You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by Otto Fowler <ot...@gmail.com> on 2017/04/25 17:30:51 UTC
Ambari Wizard: Repo Tab
Hi,
Should I see the repo tab in the ambari wizard for metron at this point?
Asking for a friend who can’t get the wizard to work :)
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
I can’t reproduce now, and the log rolled, so I’m not going to log it until
I see it again.
On April 26, 2017 at 15:47:48, Michael Miklavcic (
michael.miklavcic@gmail.com) wrote:
I thought we had addressed that at some point, so if it's blowing up still
on restart when parsers are down, then I would file a Jira.
On Wed, Apr 26, 2017 at 1:41 PM, Otto Fowler <ot...@gmail.com>
wrote:
> Ok, I did ifconfig, i mean ip a and got the right values
> that and the brackets and it looks like es is up.
>
> Now my next problem is prob. a bug.
>
> Ambari thinks it needs to restart ‘metron’.
> the enrichment restart crashes because we don’t catch the storm exception
> that you are stopping a topology that was not running.
>
> The question is - is the state wrong _and_ the exception catch or not.
>
> I am guessing I should start a Jira for that one?
>
>
>
> On April 26, 2017 at 15:31:05, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> So now I’m getting errors for ETH0
> I think that is one of the things that has to change on centos7?
> Can you check what you have?
>
>
>
> On April 26, 2017 at 15:20:18, Nick Allen (nick@nickallen.org) wrote:
>
> Yes, I am also running on CentOS 7.
>
> On Wed, Apr 26, 2017 at 3:19 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > Would i have to change that on Centos 7?
> >
> >
> > On April 26, 2017 at 15:12:38, Nick Allen (nick@nickallen.org) wrote:
> >
> > Check on the `network_hosts` setting. I just ran into the same issue. I
> > had to add brackets around it otherwise the file was not valid YAML and
> it
> > would crash trying to load the config file.
> >
> > network.host: ["_lo:ipv4_","_eth0:ipv4_"]
> >
> > On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > I think it is literally crashing trying to load the config file
> > >
> > >
> > >
> > > On April 26, 2017 at 15:00:25, Michael Miklavcic (
> > > michael.miklavcic@gmail.com) wrote:
> > >
> > > A couple more things you should check, given this setup:
> > >
> > > - 1 master node (not as data node)
> > > - 2 data nodes
> > >
> > > I'd use something like this:
> > >
> > > - gateway_recover_after_data_nodes=1 or 2
> > > - index_number_of_replicas=0 or 1 (2 data nodes, with 2 total copies
of
> > > the index - if you set this value to 2, it would need 1 more data
node
> > > to
> > > attain 1 orig index + 2 replicas)
> > >
> > >
> > >
> > > On Wed, Apr 26, 2017 at 12:02 PM, Otto Fowler <ottobackwards@gmail.com
> >
> > > wrote:
> > >
> > > > Actually never mind. I am not sure how to read the exception. Is it
> > > > failing to read the configuration?
> > > >
> > > >
> > > > On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > So the elasticsearch service is disabled. That is the issue. When
ES
> > > > calls get status ( service elasticsearch status ) it fails.
> > > > I don’t know systemd too well, but does that mean we didn’t do the
> > equiv
> > > > of chkconfig on?
> > > >
> > > >
> > > >
> > > > On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > Getting the same error in ES as before from what I can see
> > > >
> > > >
> > > > On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > I still can’t keep ES running, it dies.
> > > > Also, I’m getting exceptions from enrichment trying to restart when
> the
> > > > topology wasn’t running.
> > > >
> > > > Is there some coordination in this work? Should I just log jiras?
> > > > If someone is already sussing out ES, we can sync up
> > > >
> > > >
> > > >
> > > > On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com)
> > > wrote:
> > > >
> > > > I'm also interested to know why that's important at such a small
> scale.
> > > >
> > > > Jon
> > > >
> > > > On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com>
> > > > wrote:
> > > >
> > > > > I am following
> > > > >
> > > > > https://community.hortonworks.com/articles/60805/deploying-
> > > > a-fresh-metron-cluster-using-ambari-serv.html
> > > > > I DO have master and data node together.
> > > > > Why is that a problem?
> > > > >
> > > > > I will try again with master and data node separate.
> > > > >
> > > > >
> > > > > On April 26, 2017 at 10:41:59, Michael Miklavcic (
> > > > > michael.miklavcic@gmail.com) wrote:
> > > > >
> > > > > Hey Otto,
> > > > >
> > > > > How do you have the ES nodes configured? For a base install I
would
> > > > setup 1
> > > > > master (NOT as data node) and 2 data nodes (NOT on the same node
as
> > the
> > > > > master). This is the install configuration I got working. You can
> > also
> > > > > modify some configuration properties around master node as data
> node,
> > > > index
> > > > > replicas, and gateway recovery to get it working differently, but
> > this
> > > is
> > > > > what will work OOTB with the default config settings from the
> mpack.
> > If
> > > > > you've already setup a master node and a data node on the same
> host,
> > > > we'll
> > > > > need to re-install.
> > > > >
> > > > > Mike
> > > > >
> > > > >
> > > > > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <
> > ottobackwards@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > File
> > > > > > "/usr/lib/python2.6/site-packages/resource_management/
> > > core/shell.py",
> > > > > line
> > > > > > 293, in _call
> > > > > >
> > > > > > raise ExecutionFailed(err_msg, code, out, err)
> > > > > >
> > > > > > ExecutionFailed: Execution of 'service elasticsearch status'
> > returned
> > > > 3.
> > > > > ●
> > > > > > elasticsearch.service - Elasticsearch
> > > > > >
> > > > > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> > > > > > disabled;
> > > > > > vendor preset: disabled)
> > > > > >
> > > > > > Active: failed (Result: exit-code) since Tue 2017-04-25
22:14:58
> > EDT
> > > > > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
> > > > > >
> > > > > > Docs: http://www.elastic.co
> > > > > >
> > > > > > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/
> > elasticsearch
> > > > > > -Des.pidfile=${PID_DIR}/elasticsearch.pid
> > > -Des.default.path.home=${ES_
> > > > > > HOME}
> > > > > > -Des.default.path.logs=${LOG_DIR}
-Des.default.path.data=${DATA_
> > DIR}
> > > > > > -Des.default.path.conf=${CONF_DIR} (code=exited,
> status=1/FAILURE)
> > > > > >
> > > > > > Process: 16819
> > > > > > ExecStartPre=/usr/share/elasticsearch/bin/
> > > > elasticsearch-systemd-pre-exec
> > > > > > (code=exited, status=0/SUCCESS)
> > > > > >
> > > > > > Main PID: 16821 (code=exited, status=1/FAILURE)
> > > > > >
> > > > > >
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > > at
> > > > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > > > loadFromStream(Settings.java:1080)
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > > at
> > > > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > > > loadFromPath(Settings.java:1067)
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > > at
> > > > > > org.elasticsearch.node.internal.InternalSettingsPreparer.
> > > > > > prepareEnvironment(InternalSettingsPreparer.java:88)
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > > at
> > > > > > org.elasticsearch.bootstrap.Bootstrap.initialSettings(
> > > > Bootstrap.java:202)
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > > at
> > > > > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > > at
> > > > > > org.elasticsearch.bootstrap.Elasticsearch.main(
> > > Elasticsearch.java:35)
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > > > Refer
> > > > > > to
> > > > > > the log for complete error details.
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > > > elasticsearch.service: main process exited, code=exited,
> > > > status=1/FAILURE
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> > > > > > elasticsearch.service entered failed state.
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > > > elasticsearch.service failed.
> > > > > >
> > > > > > INFO 2017-04-26 08:55:19,962
> > > > > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
> > > > > >
> > > > > >
> > > > > > On April 26, 2017 at 08:27:27, Otto Fowler (
> > ottobackwards@gmail.com)
> > > > > > wrote:
> > > > > >
> > > > > > Can you describe what ES issues you are working on?
> > > > > > Fixing the repos got everything installed, but my ES components
> > don’t
> > > > > stay
> > > > > > running.
> > > > > > I need to harvest the errors.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On April 25, 2017 at 16:46:00, Otto Fowler (
> > ottobackwards@gmail.com)
> > > > > > wrote:
> > > > > >
> > > > > > Nm. sorry. I fixed it.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On April 25, 2017 at 16:42:05, Otto Fowler (
> > ottobackwards@gmail.com)
> > > > > > wrote:
> > > > > >
> > > > > > Ok, now I see the repos in the ‘pick version’ screen, but it is
> > > erring
> > > > on
> > > > > > the f://localrepo
> > > > > > even though the folder exists, there is no repodata/repomd.xml.
> > > > > >
> > > > > > What is the command to create a local repo?
> > > > > >
> > > > > >
> > > > > >
> > > > > > On April 25, 2017 at 16:05:17, Otto Fowler (
> > ottobackwards@gmail.com)
> > > > > > wrote:
> > > > > >
> > > > > > I was going by the HW community page.
> > > > > >
> > > > > > Ok, Let me try it
> > > > > >
> > > > > >
> > > > > >
> > > > > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com)
> > > > wrote:
> > > > > >
> > > > > > That would do it. It requires 2.4.2+. I would have sworn I put
> that
> > > in
> > > > > the
> > > > > > README, but I must have only annotated the PR. :(
> > > > > >
> > > > > > I'll get that in asap.
> > > > > >
> > > > > > -D...
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <
> > > ottobackwards@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > 2.4.1 as a matter of fact
> > > > > > >
> > > > > > >
> > > > > > > On April 25, 2017 at 15:29:43, David Lyle (
> dlyle65535@gmail.com)
> > > > > wrote:
> > > > > > >
> > > > > > > Any chance they're running Ambari < 2.4.2?
> > > > > > >
> > > > > > > -D...
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > > > > > > michael.miklavcic@gmail.com> wrote:
> > > > > > >
> > > > > > > > Hey Otto, I don't have wizard screenshots in front of me
> right
> > > now
> > > > to
> > > > > > > say
> > > > > > > > for sure, but I do see a repoinfo.xml in the mpack. I
haven't
> > run
> > > > > into
> > > > > > > > anything like that, but next time I run through the install
I
> > can
> > > > > take
> > > > > > > > another look.
> > > > > > > >
> > > > > > > > https://github.com/apache/incubator-metron/blob/master/
> > > > > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > > > > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/
> > repoinfo.xml
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
> > > > > ottobackwards@gmail.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > stderr:
> > > > > > > > >
> > > > > > > > > Traceback (most recent call last):
> > > > > > > > >
> > > > > > > > > File "/var/lib/ambari-agent/cache/
> > > common-services/ELASTICSEARCH/
> > > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in
> > <module>
> > > > > > > > >
> > > > > > > > > Elasticsearch().execute()
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > libraries/script/script.py",
> > > > > > > > > line 280, in execute
> > > > > > > > >
> > > > > > > > > method(env)
> > > > > > > > >
> > > > > > > > > File "/var/lib/ambari-agent/cache/
> > > common-services/ELASTICSEARCH/
> > > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in
> install
> > > > > > > > >
> > > > > > > > > self.install_packages(env)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > libraries/script/script.py",
> > > > > > > > > line 567, in install_packages
> > > > > > > > >
> > > > > > > > > retry_count=agent_stack_retry_count)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > core/base.py",
> > > > > > > > > line 155, in __init__
> > > > > > > > >
> > > > > > > > > self.env.run()
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > core/environment.py",
> > > > > > > > > line 160, in run
> > > > > > > > >
> > > > > > > > > self.run_action(resource, action)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > core/environment.py",
> > > > > > > > > line 124, in run_action
> > > > > > > > >
> > > > > > > > > provider_action()
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > > core/providers/package/__init__.py", line 54, in
> > > action_install
> > > > > > > > >
> > > > > > > > > self.install_package(package_name,
> self.resource.use_repos,
> > > > > > > > > self.resource.skip_repos)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > > core/providers/package/yumrpm.py", line 49, in
> > install_package
> > > > > > > > >
> > > > > > > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > > > > > > logoutput=self.get_logoutput())
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > > core/providers/package/__init__.py", line 83, in
> > > > > > > > checked_call_with_retries
> > > > > > > > >
> > > > > > > > > return self._call_with_retries(cmd, is_checked=True,
> > **kwargs)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > > core/providers/package/__init__.py", line 91, in
> > > > _call_with_retries
> > > > > > > > >
> > > > > > > > > code, out = func(cmd, **kwargs)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > core/shell.py",
> > > > > > > > > line 71, in inner
> > > > > > > > >
> > > > > > > > > result = function(command, **kwargs)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > core/shell.py",
> > > > > > > > > line 93, in checked_call
> > > > > > > > >
> > > > > > > > > tries=tries, try_sleep=try_sleep)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > core/shell.py",
> > > > > > > > > line 141, in _call_wrapper
> > > > > > > > >
> > > > > > > > > result = _call(command, **kwargs_copy)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > core/shell.py",
> > > > > > > > > line 294, in _call
> > > > > > > > >
> > > > > > > > > raise Fail(err_msg)
> > > > > > > > >
> > > > > > > > > resource_management.core.exceptions.Fail: Execution of
> > > > > '/usr/bin/yum
> > > > > > > -d
> > > > > > > > 0
> > > > > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error:
> > Nothing
> > > > to
> > > > > do
> > > > > > > > >
> > > > > > > > > stdout:
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,669
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Using
> > > > > hadoop
> > > > > > > > conf
> > > > > > > > > dir: /usr/hdp/current/hadoop-client/conf
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,671
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['metron']
> > > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,673
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['metron']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,740
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['livy']
> > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,741
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['livy']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,774
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > > > Group['elasticsearch'] {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,775
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['elasticsearch']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,812
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['spark']
> > > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,812
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['spark']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,848
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > > Group['zeppelin']
> > > > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,849
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['zeppelin']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,884
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['hadoop']
> > > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,885
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['hadoop']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,921
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['kibana']
> > > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,922
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['kibana']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,959
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['users']
> > > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,960
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > User['hive']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,961
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['hive']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,227
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['storm']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,227
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['storm']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,388
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > > User['zookeeper']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,388
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['zookeeper']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,518
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > > > User['infra-solr'] {'gid': 'hadoop',
> 'fetch_nonlocal_groups':
> > > > True,
> > > > > > > > > 'groups': [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,518
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['infra-solr']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,578
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['ams']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,578
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['ams']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,625
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['tez']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > [u'users']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,626
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['tez']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,660
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['zeppelin']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,660
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['zeppelin']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,705
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['metron']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,705
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['metron']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,760
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['livy']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,760
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['livy']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,803
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > > > User['elasticsearch'] {'gid': 'hadoop',
> > > 'fetch_nonlocal_groups':
> > > > > > True,
> > > > > > > > > 'groups': [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,804
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['elasticsearch']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,837
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['spark']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,837
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['spark']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,885
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > > User['ambari-qa']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > [u'users']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,885
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['ambari-qa']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,928
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['flume']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,928
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['flume']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,972
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['kafka']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,973
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['kafka']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,025
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['hdfs']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,026
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['hdfs']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,075
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['yarn']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,075
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['yarn']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,125
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > User['kibana']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,126
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['kibana']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,175
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > User['mapred']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,176
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['mapred']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,223
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > User['hbase']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,224
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['hbase']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,271
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['hcat']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,272
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['hcat']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,320
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh']
{'content':
> > > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,333
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Writing
> > > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it
> > > > doesn't
> > > > > > > exist
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,334
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Changing
> > > > > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh
from
> > 644
> > > > to
> > > > > > 555
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,335
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa']
{'not_if':
> > > > '(test
> > > > > > > $(id
> > > > > > > > -u
> > > > > > > > > ambari-qa) -gt 1000) || (false)'}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,364
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Skipping
> > > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to
> > not_if
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,365
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
> > > > 'create_parents':
> > > > > > > True,
> > > > > > > > > 'mode': 0775, 'cd_access': 'a'}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,366
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Creating
> > > > > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't
> > exist.
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,366
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Changing
> > > > > > > owner
> > > > > > > > > for /tmp/hbase-hbase from 0 to hbase
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,366
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Changing
> > > > > > > > > permission for /tmp/hbase-hbase from 755 to 775
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,368
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh']
{'content':
> > > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,369
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > > > >
> > > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > > > hbase-hbase']
> > > > > > >
> > > > > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,393
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Skipping
> > > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > > > >
> > > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > > > hbase-hbase']
> > > > > > >
> > > > > > > > > due to not_if
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,394
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > Group['hdfs']
> > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,395
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['hdfs']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,430
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['hdfs']
> > > > > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop',
> > u'hdfs']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,431
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Modifying
> > > > > > > user
> > > > > > > > > hdfs
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,474
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
FS
> > > Type:
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,474
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,474
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Creating
> > > > > > > > > directory Directory['/etc/hadoop'] since it doesn't
exist.
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,475
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
> tmpdir']
> > > > > > > {'owner':
> > > > > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,476
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Creating
> > > > > > > > > directory Directory['/var/lib/ambari-
> > agent/tmp/hadoop_java_io_
> > > > > > tmpdir']
> > > > > > >
> > > > > > > > > since it doesn't exist.
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,476
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Changing
> > > > > > > owner
> > > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
0
> > to
> > > > hdfs
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,476
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Changing
> > > > > > > group
> > > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
0
> > to
> > > > > hadoop
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,477
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Changing
> > > > > > > > > permission for /var/lib/ambari-agent/tmp/
> > hadoop_java_io_tmpdir
> > > > from
> > > > > > > 755
> > > > > > > > > to 1777
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,477
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > > > > > > {'create_parents': True}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,478
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Creating
> > > > > > > > > directory Directory['/var/lib/ambari-
> > > > agent/tmp/AMBARI-artifacts/']
> > > > > > > since
> > > > > > > > > it doesn't exist.
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,479
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > >
File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'
> ]
> > > > > > > {'content':
> > > > > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,504
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Downloading
> > > > > > > the
> > > > > > > > > file from http://css-fx-met-poc-82:8080/
> resources//jdk-8u77-
> > > > > > > > > linux-x64.tar.gz
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:53,793
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > > > > Directory['/usr/jdk64'] {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:53,793
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > Creating
> > > > > > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:53,794
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > > > Execute[('chmod',
> > > > > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:53,824
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > Execute['cd
> > > > > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> > > > > > ambari-sudo.sh
> > > > > > > cp
> > > > > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/*
> /usr/jdk64']
> > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,439
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
> > > {'action':
> > > > > > > > > ['delete']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,440
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Removing
> > > > > > > > > directory Directory['/var/lib/ambari-
> > > agent/tmp/jdk_tmp_1BRMsS']
> > > > and
> > > > > > > all
> > > > > > > > > its content
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,620
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
> > > > 'cd_access':
> > > > > > > 'a'}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,621
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > Execute[('chmod',
> > > > > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,682
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Initializing
> > > > > > > 2
> > > > > > > > > repositories
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,683
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0',
> 'action':
> > > > > > > ['create'],
> > > > > > > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > > > %}mirrorlist={{mirror_list}}{% else
> %}baseurl={{base_url}}{%
> > > > endif
> > > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> > 'HDP',
> > > > > > > > > 'mirror_list': None}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,709
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content':
> > > > InlineTemplate(...)}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,711
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Writing
> > > > > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't
exist
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,711
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
> > > > > http://public-repo-1.
> > > > >
> > > > > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7',
> 'action':
> > > > > > > ['create'],
> > > > > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > > > %}mirrorlist={{mirror_list}}{% else
> %}baseurl={{base_url}}{%
> > > > endif
> > > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> > > > > 'HDP-UTILS',
> > > > > > > > > 'mirror_list': None}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,717
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > > > > > > InlineTemplate(...)}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,717
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Writing
> > > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it
doesn't
> > > exist
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,718
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > Package['unzip']
> > > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,846
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Installing
> > > > > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:03,413
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > > > Package['curl']
> > > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:03,429
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Skipping
> > > > > > > > > installation of existing package curl
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:03,430
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > > > > > Package['hdp-select'] {'retry_on_repo_unavailability':
> > False,
> > > > > > > > > 'retry_count': 5}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:03,451
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > Installing
> > > > > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
> > > > hdp-select')
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:04,948
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > Install
> > > > ES
> > > > > > > Data
> > > > > > > > > Node
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:04,952
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_
> > > unavailability':
> > > > > > > False,
> > > > > > > > > 'retry_count': 5}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:05,086
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> -
> > > > Installing
> > > > > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y
> > install
> > > > > > > > > elasticsearch-2.3.3')
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Command failed after 1 tries
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
> > > > > ottobackwards@gmail.com)
> > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Yes, I’m going to run through again and get the logs. The
> > basic
> > > > > issue
> > > > > > > > is,
> > > > > > > > > no repo tab -> if I should have one,
> > > > > > > > > and the ES yum command failing with nothing to do, which
> > stops
> > > > > > > deployment
> > > > > > > > > on all the nodes.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > > > > > > michael.miklavcic@gmail.com) wrote:
> > > > > > > > >
> > > > > > > > > I've been doing an internal hwx deployment (HCP) recently
> for
> > > > > testing
> > > > > > > > > purposes. I have some ES tweaks to work out, but other
than
> > > that
> > > > it
> > > > > > > has
> > > > > > > > > been running smoothly. Can you share any log info or
> > > screenshots?
> > > > > > > > >
> > > > > > > > > Mike
> > > > > > > > >
> > > > > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <
> > > ottobackwards@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > > Should I see the repo tab in the ambari wizard for
metron
> > at
> > > > this
> > > > > > > > point?
> > > > > > > > > > Asking for a friend who can’t get the wizard to work :)
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > --
> > > >
> > > > Jon
> > > >
> > > >
> > >
> >
> >
>
Re: Ambari Wizard: Repo Tab
Posted by Michael Miklavcic <mi...@gmail.com>.
I thought we had addressed that at some point, so if it's blowing up still
on restart when parsers are down, then I would file a Jira.
On Wed, Apr 26, 2017 at 1:41 PM, Otto Fowler <ot...@gmail.com>
wrote:
> Ok, I did ifconfig, i mean ip a and got the right values
> that and the brackets and it looks like es is up.
>
> Now my next problem is prob. a bug.
>
> Ambari thinks it needs to restart ‘metron’.
> the enrichment restart crashes because we don’t catch the storm exception
> that you are stopping a topology that was not running.
>
> The question is - is the state wrong _and_ the exception catch or not.
>
> I am guessing I should start a Jira for that one?
>
>
>
> On April 26, 2017 at 15:31:05, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> So now I’m getting errors for ETH0
> I think that is one of the things that has to change on centos7?
> Can you check what you have?
>
>
>
> On April 26, 2017 at 15:20:18, Nick Allen (nick@nickallen.org) wrote:
>
> Yes, I am also running on CentOS 7.
>
> On Wed, Apr 26, 2017 at 3:19 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > Would i have to change that on Centos 7?
> >
> >
> > On April 26, 2017 at 15:12:38, Nick Allen (nick@nickallen.org) wrote:
> >
> > Check on the `network_hosts` setting. I just ran into the same issue. I
> > had to add brackets around it otherwise the file was not valid YAML and
> it
> > would crash trying to load the config file.
> >
> > network.host: ["_lo:ipv4_","_eth0:ipv4_"]
> >
> > On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > I think it is literally crashing trying to load the config file
> > >
> > >
> > >
> > > On April 26, 2017 at 15:00:25, Michael Miklavcic (
> > > michael.miklavcic@gmail.com) wrote:
> > >
> > > A couple more things you should check, given this setup:
> > >
> > > - 1 master node (not as data node)
> > > - 2 data nodes
> > >
> > > I'd use something like this:
> > >
> > > - gateway_recover_after_data_nodes=1 or 2
> > > - index_number_of_replicas=0 or 1 (2 data nodes, with 2 total copies of
> > > the index - if you set this value to 2, it would need 1 more data node
> > > to
> > > attain 1 orig index + 2 replicas)
> > >
> > >
> > >
> > > On Wed, Apr 26, 2017 at 12:02 PM, Otto Fowler <ottobackwards@gmail.com
> >
> > > wrote:
> > >
> > > > Actually never mind. I am not sure how to read the exception. Is it
> > > > failing to read the configuration?
> > > >
> > > >
> > > > On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > So the elasticsearch service is disabled. That is the issue. When ES
> > > > calls get status ( service elasticsearch status ) it fails.
> > > > I don’t know systemd too well, but does that mean we didn’t do the
> > equiv
> > > > of chkconfig on?
> > > >
> > > >
> > > >
> > > > On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > Getting the same error in ES as before from what I can see
> > > >
> > > >
> > > > On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > I still can’t keep ES running, it dies.
> > > > Also, I’m getting exceptions from enrichment trying to restart when
> the
> > > > topology wasn’t running.
> > > >
> > > > Is there some coordination in this work? Should I just log jiras?
> > > > If someone is already sussing out ES, we can sync up
> > > >
> > > >
> > > >
> > > > On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com)
> > > wrote:
> > > >
> > > > I'm also interested to know why that's important at such a small
> scale.
> > > >
> > > > Jon
> > > >
> > > > On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com>
> > > > wrote:
> > > >
> > > > > I am following
> > > > >
> > > > > https://community.hortonworks.com/articles/60805/deploying-
> > > > a-fresh-metron-cluster-using-ambari-serv.html
> > > > > I DO have master and data node together.
> > > > > Why is that a problem?
> > > > >
> > > > > I will try again with master and data node separate.
> > > > >
> > > > >
> > > > > On April 26, 2017 at 10:41:59, Michael Miklavcic (
> > > > > michael.miklavcic@gmail.com) wrote:
> > > > >
> > > > > Hey Otto,
> > > > >
> > > > > How do you have the ES nodes configured? For a base install I would
> > > > setup 1
> > > > > master (NOT as data node) and 2 data nodes (NOT on the same node as
> > the
> > > > > master). This is the install configuration I got working. You can
> > also
> > > > > modify some configuration properties around master node as data
> node,
> > > > index
> > > > > replicas, and gateway recovery to get it working differently, but
> > this
> > > is
> > > > > what will work OOTB with the default config settings from the
> mpack.
> > If
> > > > > you've already setup a master node and a data node on the same
> host,
> > > > we'll
> > > > > need to re-install.
> > > > >
> > > > > Mike
> > > > >
> > > > >
> > > > > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <
> > ottobackwards@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > File
> > > > > > "/usr/lib/python2.6/site-packages/resource_management/
> > > core/shell.py",
> > > > > line
> > > > > > 293, in _call
> > > > > >
> > > > > > raise ExecutionFailed(err_msg, code, out, err)
> > > > > >
> > > > > > ExecutionFailed: Execution of 'service elasticsearch status'
> > returned
> > > > 3.
> > > > > ●
> > > > > > elasticsearch.service - Elasticsearch
> > > > > >
> > > > > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> > > > > > disabled;
> > > > > > vendor preset: disabled)
> > > > > >
> > > > > > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58
> > EDT
> > > > > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
> > > > > >
> > > > > > Docs: http://www.elastic.co
> > > > > >
> > > > > > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/
> > elasticsearch
> > > > > > -Des.pidfile=${PID_DIR}/elasticsearch.pid
> > > -Des.default.path.home=${ES_
> > > > > > HOME}
> > > > > > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_
> > DIR}
> > > > > > -Des.default.path.conf=${CONF_DIR} (code=exited,
> status=1/FAILURE)
> > > > > >
> > > > > > Process: 16819
> > > > > > ExecStartPre=/usr/share/elasticsearch/bin/
> > > > elasticsearch-systemd-pre-exec
> > > > > > (code=exited, status=0/SUCCESS)
> > > > > >
> > > > > > Main PID: 16821 (code=exited, status=1/FAILURE)
> > > > > >
> > > > > >
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > > at
> > > > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > > > loadFromStream(Settings.java:1080)
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > > at
> > > > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > > > loadFromPath(Settings.java:1067)
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > > at
> > > > > > org.elasticsearch.node.internal.InternalSettingsPreparer.
> > > > > > prepareEnvironment(InternalSettingsPreparer.java:88)
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > > at
> > > > > > org.elasticsearch.bootstrap.Bootstrap.initialSettings(
> > > > Bootstrap.java:202)
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > > at
> > > > > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > > at
> > > > > > org.elasticsearch.bootstrap.Elasticsearch.main(
> > > Elasticsearch.java:35)
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > > > Refer
> > > > > > to
> > > > > > the log for complete error details.
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > > > elasticsearch.service: main process exited, code=exited,
> > > > status=1/FAILURE
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> > > > > > elasticsearch.service entered failed state.
> > > > > >
> > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> > 04-25%2022:14:58%20EDT
> > > >
> > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > > > elasticsearch.service failed.
> > > > > >
> > > > > > INFO 2017-04-26 08:55:19,962
> > > > > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
> > > > > >
> > > > > >
> > > > > > On April 26, 2017 at 08:27:27, Otto Fowler (
> > ottobackwards@gmail.com)
> > > > > > wrote:
> > > > > >
> > > > > > Can you describe what ES issues you are working on?
> > > > > > Fixing the repos got everything installed, but my ES components
> > don’t
> > > > > stay
> > > > > > running.
> > > > > > I need to harvest the errors.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On April 25, 2017 at 16:46:00, Otto Fowler (
> > ottobackwards@gmail.com)
> > > > > > wrote:
> > > > > >
> > > > > > Nm. sorry. I fixed it.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On April 25, 2017 at 16:42:05, Otto Fowler (
> > ottobackwards@gmail.com)
> > > > > > wrote:
> > > > > >
> > > > > > Ok, now I see the repos in the ‘pick version’ screen, but it is
> > > erring
> > > > on
> > > > > > the f://localrepo
> > > > > > even though the folder exists, there is no repodata/repomd.xml.
> > > > > >
> > > > > > What is the command to create a local repo?
> > > > > >
> > > > > >
> > > > > >
> > > > > > On April 25, 2017 at 16:05:17, Otto Fowler (
> > ottobackwards@gmail.com)
> > > > > > wrote:
> > > > > >
> > > > > > I was going by the HW community page.
> > > > > >
> > > > > > Ok, Let me try it
> > > > > >
> > > > > >
> > > > > >
> > > > > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com)
> > > > wrote:
> > > > > >
> > > > > > That would do it. It requires 2.4.2+. I would have sworn I put
> that
> > > in
> > > > > the
> > > > > > README, but I must have only annotated the PR. :(
> > > > > >
> > > > > > I'll get that in asap.
> > > > > >
> > > > > > -D...
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <
> > > ottobackwards@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > 2.4.1 as a matter of fact
> > > > > > >
> > > > > > >
> > > > > > > On April 25, 2017 at 15:29:43, David Lyle (
> dlyle65535@gmail.com)
> > > > > wrote:
> > > > > > >
> > > > > > > Any chance they're running Ambari < 2.4.2?
> > > > > > >
> > > > > > > -D...
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > > > > > > michael.miklavcic@gmail.com> wrote:
> > > > > > >
> > > > > > > > Hey Otto, I don't have wizard screenshots in front of me
> right
> > > now
> > > > to
> > > > > > > say
> > > > > > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't
> > run
> > > > > into
> > > > > > > > anything like that, but next time I run through the install I
> > can
> > > > > take
> > > > > > > > another look.
> > > > > > > >
> > > > > > > > https://github.com/apache/incubator-metron/blob/master/
> > > > > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > > > > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/
> > repoinfo.xml
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
> > > > > ottobackwards@gmail.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > stderr:
> > > > > > > > >
> > > > > > > > > Traceback (most recent call last):
> > > > > > > > >
> > > > > > > > > File "/var/lib/ambari-agent/cache/
> > > common-services/ELASTICSEARCH/
> > > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in
> > <module>
> > > > > > > > >
> > > > > > > > > Elasticsearch().execute()
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > libraries/script/script.py",
> > > > > > > > > line 280, in execute
> > > > > > > > >
> > > > > > > > > method(env)
> > > > > > > > >
> > > > > > > > > File "/var/lib/ambari-agent/cache/
> > > common-services/ELASTICSEARCH/
> > > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in
> install
> > > > > > > > >
> > > > > > > > > self.install_packages(env)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > libraries/script/script.py",
> > > > > > > > > line 567, in install_packages
> > > > > > > > >
> > > > > > > > > retry_count=agent_stack_retry_count)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > core/base.py",
> > > > > > > > > line 155, in __init__
> > > > > > > > >
> > > > > > > > > self.env.run()
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > core/environment.py",
> > > > > > > > > line 160, in run
> > > > > > > > >
> > > > > > > > > self.run_action(resource, action)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > core/environment.py",
> > > > > > > > > line 124, in run_action
> > > > > > > > >
> > > > > > > > > provider_action()
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > > core/providers/package/__init__.py", line 54, in
> > > action_install
> > > > > > > > >
> > > > > > > > > self.install_package(package_name,
> self.resource.use_repos,
> > > > > > > > > self.resource.skip_repos)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > > core/providers/package/yumrpm.py", line 49, in
> > install_package
> > > > > > > > >
> > > > > > > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > > > > > > logoutput=self.get_logoutput())
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > > core/providers/package/__init__.py", line 83, in
> > > > > > > > checked_call_with_retries
> > > > > > > > >
> > > > > > > > > return self._call_with_retries(cmd, is_checked=True,
> > **kwargs)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > > core/providers/package/__init__.py", line 91, in
> > > > _call_with_retries
> > > > > > > > >
> > > > > > > > > code, out = func(cmd, **kwargs)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > core/shell.py",
> > > > > > > > > line 71, in inner
> > > > > > > > >
> > > > > > > > > result = function(command, **kwargs)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > core/shell.py",
> > > > > > > > > line 93, in checked_call
> > > > > > > > >
> > > > > > > > > tries=tries, try_sleep=try_sleep)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > core/shell.py",
> > > > > > > > > line 141, in _call_wrapper
> > > > > > > > >
> > > > > > > > > result = _call(command, **kwargs_copy)
> > > > > > > > >
> > > > > > > > > File "/usr/lib/python2.6/site-
> packages/resource_management/
> > > > > > > > core/shell.py",
> > > > > > > > > line 294, in _call
> > > > > > > > >
> > > > > > > > > raise Fail(err_msg)
> > > > > > > > >
> > > > > > > > > resource_management.core.exceptions.Fail: Execution of
> > > > > '/usr/bin/yum
> > > > > > > -d
> > > > > > > > 0
> > > > > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error:
> > Nothing
> > > > to
> > > > > do
> > > > > > > > >
> > > > > > > > > stdout:
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,669
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Using
> > > > > hadoop
> > > > > > > > conf
> > > > > > > > > dir: /usr/hdp/current/hadoop-client/conf
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,671
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['metron']
> > > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,673
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['metron']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,740
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['livy']
> > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,741
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['livy']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,774
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > > > Group['elasticsearch'] {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,775
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['elasticsearch']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,812
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['spark']
> > > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,812
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['spark']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,848
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > > Group['zeppelin']
> > > > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,849
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['zeppelin']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,884
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['hadoop']
> > > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,885
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['hadoop']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,921
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['kibana']
> > > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,922
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['kibana']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,959
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['users']
> > > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,960
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > User['hive']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:48,961
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['hive']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,227
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['storm']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,227
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['storm']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,388
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > > User['zookeeper']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,388
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['zookeeper']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,518
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > > > User['infra-solr'] {'gid': 'hadoop',
> 'fetch_nonlocal_groups':
> > > > True,
> > > > > > > > > 'groups': [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,518
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['infra-solr']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,578
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['ams']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,578
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['ams']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,625
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['tez']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'users']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,626
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['tez']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,660
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['zeppelin']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,660
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['zeppelin']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,705
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['metron']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,705
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['metron']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,760
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['livy']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,760
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['livy']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,803
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > > > User['elasticsearch'] {'gid': 'hadoop',
> > > 'fetch_nonlocal_groups':
> > > > > > True,
> > > > > > > > > 'groups': [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,804
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['elasticsearch']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,837
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['spark']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,837
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['spark']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,885
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > > User['ambari-qa']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'users']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,885
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['ambari-qa']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,928
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['flume']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,928
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['flume']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,972
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['kafka']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:49,973
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['kafka']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,025
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['hdfs']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,026
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['hdfs']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,075
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['yarn']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,075
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['yarn']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,125
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > User['kibana']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,126
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['kibana']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,175
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > User['mapred']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,176
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['mapred']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,223
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > User['hbase']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,224
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['hbase']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,271
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['hcat']
> > > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > > [u'hadoop']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,272
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Adding
> > > > > user
> > > > > > > > > User['hcat']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,320
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,333
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Writing
> > > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it
> > > > doesn't
> > > > > > > exist
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,334
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Changing
> > > > > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from
> > 644
> > > > to
> > > > > > 555
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,335
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if':
> > > > '(test
> > > > > > > $(id
> > > > > > > > -u
> > > > > > > > > ambari-qa) -gt 1000) || (false)'}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,364
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Skipping
> > > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to
> > not_if
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,365
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
> > > > 'create_parents':
> > > > > > > True,
> > > > > > > > > 'mode': 0775, 'cd_access': 'a'}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,366
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Creating
> > > > > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't
> > exist.
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,366
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Changing
> > > > > > > owner
> > > > > > > > > for /tmp/hbase-hbase from 0 to hbase
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,366
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Changing
> > > > > > > > > permission for /tmp/hbase-hbase from 755 to 775
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,368
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,369
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > > > >
> > > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > > > hbase-hbase']
> > > > > > >
> > > > > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,393
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Skipping
> > > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > > > >
> > > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > > > hbase-hbase']
> > > > > > >
> > > > > > > > > due to not_if
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,394
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > Group['hdfs']
> > > > > > > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,395
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Adding
> > > > > group
> > > > > > > > > Group['hdfs']
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,430
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['hdfs']
> > > > > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop',
> > u'hdfs']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,431
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Modifying
> > > > > > > user
> > > > > > > > > hdfs
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,474
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS
> > > Type:
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,474
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,474
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Creating
> > > > > > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,475
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
> tmpdir']
> > > > > > > {'owner':
> > > > > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,476
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Creating
> > > > > > > > > directory Directory['/var/lib/ambari-
> > agent/tmp/hadoop_java_io_
> > > > > > tmpdir']
> > > > > > >
> > > > > > > > > since it doesn't exist.
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,476
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Changing
> > > > > > > owner
> > > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0
> > to
> > > > hdfs
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,476
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Changing
> > > > > > > group
> > > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0
> > to
> > > > > hadoop
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,477
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Changing
> > > > > > > > > permission for /var/lib/ambari-agent/tmp/
> > hadoop_java_io_tmpdir
> > > > from
> > > > > > > 755
> > > > > > > > > to 1777
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,477
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > > > > > > {'create_parents': True}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,478
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Creating
> > > > > > > > > directory Directory['/var/lib/ambari-
> > > > agent/tmp/AMBARI-artifacts/']
> > > > > > > since
> > > > > > > > > it doesn't exist.
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,479
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'
> ]
> > > > > > > {'content':
> > > > > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:50,504
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Downloading
> > > > > > > the
> > > > > > > > > file from http://css-fx-met-poc-82:8080/
> resources//jdk-8u77-
> > > > > > > > > linux-x64.tar.gz
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:53,793
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > > > > Directory['/usr/jdk64'] {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:53,793
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > Creating
> > > > > > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:53,794
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > > > Execute[('chmod',
> > > > > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:12:53,824
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > Execute['cd
> > > > > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> > > > > > ambari-sudo.sh
> > > > > > > cp
> > > > > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/*
> /usr/jdk64']
> > {}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,439
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
> > > {'action':
> > > > > > > > > ['delete']}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,440
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Removing
> > > > > > > > > directory Directory['/var/lib/ambari-
> > > agent/tmp/jdk_tmp_1BRMsS']
> > > > and
> > > > > > > all
> > > > > > > > > its content
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,620
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
> > > > 'cd_access':
> > > > > > > 'a'}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,621
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > Execute[('chmod',
> > > > > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,682
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Initializing
> > > > > > > 2
> > > > > > > > > repositories
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,683
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0',
> 'action':
> > > > > > > ['create'],
> > > > > > > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > > > %}mirrorlist={{mirror_list}}{% else
> %}baseurl={{base_url}}{%
> > > > endif
> > > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> > 'HDP',
> > > > > > > > > 'mirror_list': None}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,709
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content':
> > > > InlineTemplate(...)}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,711
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Writing
> > > > > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,711
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
> > > > > http://public-repo-1.
> > > > >
> > > > > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7',
> 'action':
> > > > > > > ['create'],
> > > > > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > > > %}mirrorlist={{mirror_list}}{% else
> %}baseurl={{base_url}}{%
> > > > endif
> > > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> > > > > 'HDP-UTILS',
> > > > > > > > > 'mirror_list': None}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,717
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > > > > > > InlineTemplate(...)}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,717
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Writing
> > > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't
> > > exist
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,718
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > Package['unzip']
> > > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:00,846
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Installing
> > > > > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:03,413
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > > > Package['curl']
> > > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:03,429
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Skipping
> > > > > > > > > installation of existing package curl
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:03,430
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > > > > > Package['hdp-select'] {'retry_on_repo_unavailability':
> > False,
> > > > > > > > > 'retry_count': 5}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:03,451
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > Installing
> > > > > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
> > > > hdp-select')
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:04,948
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > Install
> > > > ES
> > > > > > > Data
> > > > > > > > > Node
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:04,952
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_
> > > unavailability':
> > > > > > > False,
> > > > > > > > > 'retry_count': 5}
> > > > > > > > >
> > > > > > > > > 2017-04-25 14:13:05,086
> > > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> -
> > > > Installing
> > > > > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y
> > install
> > > > > > > > > elasticsearch-2.3.3')
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Command failed after 1 tries
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
> > > > > ottobackwards@gmail.com)
> > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Yes, I’m going to run through again and get the logs. The
> > basic
> > > > > issue
> > > > > > > > is,
> > > > > > > > > no repo tab -> if I should have one,
> > > > > > > > > and the ES yum command failing with nothing to do, which
> > stops
> > > > > > > deployment
> > > > > > > > > on all the nodes.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > > > > > > michael.miklavcic@gmail.com) wrote:
> > > > > > > > >
> > > > > > > > > I've been doing an internal hwx deployment (HCP) recently
> for
> > > > > testing
> > > > > > > > > purposes. I have some ES tweaks to work out, but other than
> > > that
> > > > it
> > > > > > > has
> > > > > > > > > been running smoothly. Can you share any log info or
> > > screenshots?
> > > > > > > > >
> > > > > > > > > Mike
> > > > > > > > >
> > > > > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <
> > > ottobackwards@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > > Should I see the repo tab in the ambari wizard for metron
> > at
> > > > this
> > > > > > > > point?
> > > > > > > > > > Asking for a friend who can’t get the wizard to work :)
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > --
> > > >
> > > > Jon
> > > >
> > > >
> > >
> >
> >
>
Re: Ambari Wizard: Repo Tab
Posted by Nick Allen <ni...@nickallen.org>.
It is actually pretty handy. You can get those type of logs right in the
Ambari web interface. If you click on "Ops" near the top of the Ambari
screen, it will open a window where you can dig-into all the previous
operations that were run. From there you can find the operation that
failed and click through to get specific log messages from any host.
Ambari user documentation is pretty good. No comment on Mpack programming
documentation. :)
On Wed, Apr 26, 2017 at 5:06 PM, Otto Fowler <ot...@gmail.com>
wrote:
> Thanks Nick. I don’t think we catch exceptions at all from the execute
> calls. I wonder what the best practice is?
> I can’t seem to find the ambari mpack programming documentation ;)
>
>
> On April 26, 2017 at 16:50:18, Nick Allen (nick@nickallen.org) wrote:
>
> I can create the JIRA. I capture the logs.
>
> On Wed, Apr 26, 2017 at 4:48 PM, Nick Allen <ni...@nickallen.org> wrote:
>
>> Yes, Otto. I just experienced that myself. It is a bug that we should
>> create a JIRA for.
>>
>> I was able to work around it by just using "start" instead of "restart".
>>
>> On Wed, Apr 26, 2017 at 3:41 PM, Otto Fowler <ot...@gmail.com>
>> wrote:
>>
>>> Ok, I did ifconfig, i mean ip a and got the right values
>>> that and the brackets and it looks like es is up.
>>>
>>> Now my next problem is prob. a bug.
>>>
>>> Ambari thinks it needs to restart ‘metron’.
>>> the enrichment restart crashes because we don’t catch the storm
>>> exception that you are stopping a topology that was not running.
>>>
>>> The question is - is the state wrong _and_ the exception catch or not.
>>>
>>> I am guessing I should start a Jira for that one?
>>>
>>>
>>>
>>> On April 26, 2017 at 15:31:05, Otto Fowler (ottobackwards@gmail.com)
>>> wrote:
>>>
>>> So now I’m getting errors for ETH0
>>> I think that is one of the things that has to change on centos7?
>>> Can you check what you have?
>>>
>>>
>>>
>>> On April 26, 2017 at 15:20:18, Nick Allen (nick@nickallen.org) wrote:
>>>
>>> Yes, I am also running on CentOS 7.
>>>
>>> On Wed, Apr 26, 2017 at 3:19 PM, Otto Fowler <ot...@gmail.com>
>>> wrote:
>>>
>>>> Would i have to change that on Centos 7?
>>>>
>>>>
>>>> On April 26, 2017 at 15:12:38, Nick Allen (nick@nickallen.org) wrote:
>>>>
>>>> Check on the `network_hosts` setting. I just ran into the same issue. I
>>>> had to add brackets around it otherwise the file was not valid YAML and
>>>> it
>>>> would crash trying to load the config file.
>>>>
>>>> network.host: ["_lo:ipv4_","_eth0:ipv4_"]
>>>>
>>>> On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler <ot...@gmail.com>
>>>> wrote:
>>>>
>>>> > I think it is literally crashing trying to load the config file
>>>> >
>>>> >
>>>> >
>>>> > On April 26, 2017 at 15:00:25, Michael Miklavcic (
>>>> > michael.miklavcic@gmail.com) wrote:
>>>> >
>>>> > A couple more things you should check, given this setup:
>>>> >
>>>> > - 1 master node (not as data node)
>>>> > - 2 data nodes
>>>> >
>>>> > I'd use something like this:
>>>> >
>>>> > - gateway_recover_after_data_nodes=1 or 2
>>>> > - index_number_of_replicas=0 or 1 (2 data nodes, with 2 total copies
>>>> of
>>>> > the index - if you set this value to 2, it would need 1 more data node
>>>> > to
>>>> > attain 1 orig index + 2 replicas)
>>>> >
>>>> >
>>>> >
>>>> > On Wed, Apr 26, 2017 at 12:02 PM, Otto Fowler <
>>>> ottobackwards@gmail.com>
>>>> > wrote:
>>>> >
>>>> > > Actually never mind. I am not sure how to read the exception. Is it
>>>> > > failing to read the configuration?
>>>> > >
>>>> > >
>>>> > > On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwards@gmail.com
>>>> )
>>>> > > wrote:
>>>> > >
>>>> > > So the elasticsearch service is disabled. That is the issue. When ES
>>>> > > calls get status ( service elasticsearch status ) it fails.
>>>> > > I don’t know systemd too well, but does that mean we didn’t do the
>>>> equiv
>>>> > > of chkconfig on?
>>>> > >
>>>> > >
>>>> > >
>>>> > > On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com
>>>> )
>>>> > > wrote:
>>>> > >
>>>> > > Getting the same error in ES as before from what I can see
>>>> > >
>>>> > >
>>>> > > On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com
>>>> )
>>>> > > wrote:
>>>> > >
>>>> > > I still can’t keep ES running, it dies.
>>>> > > Also, I’m getting exceptions from enrichment trying to restart when
>>>> the
>>>> > > topology wasn’t running.
>>>> > >
>>>> > > Is there some coordination in this work? Should I just log jiras?
>>>> > > If someone is already sussing out ES, we can sync up
>>>> > >
>>>> > >
>>>> > >
>>>> > > On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com)
>>>> > wrote:
>>>> > >
>>>> > > I'm also interested to know why that's important at such a small
>>>> scale.
>>>> > >
>>>> > > Jon
>>>> > >
>>>> > > On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ottobackwards@gmail.com
>>>> >
>>>> > > wrote:
>>>> > >
>>>> > > > I am following
>>>> > > >
>>>> > > > https://community.hortonworks.com/articles/60805/deploying-
>>>> > > a-fresh-metron-cluster-using-ambari-serv.html
>>>> > > > I DO have master and data node together.
>>>> > > > Why is that a problem?
>>>> > > >
>>>> > > > I will try again with master and data node separate.
>>>> > > >
>>>> > > >
>>>> > > > On April 26, 2017 at 10:41:59, Michael Miklavcic (
>>>> > > > michael.miklavcic@gmail.com) wrote:
>>>> > > >
>>>> > > > Hey Otto,
>>>> > > >
>>>> > > > How do you have the ES nodes configured? For a base install I
>>>> would
>>>> > > setup 1
>>>> > > > master (NOT as data node) and 2 data nodes (NOT on the same node
>>>> as the
>>>> > > > master). This is the install configuration I got working. You can
>>>> also
>>>> > > > modify some configuration properties around master node as data
>>>> node,
>>>> > > index
>>>> > > > replicas, and gateway recovery to get it working differently, but
>>>> this
>>>> > is
>>>> > > > what will work OOTB with the default config settings from the
>>>> mpack. If
>>>> > > > you've already setup a master node and a data node on the same
>>>> host,
>>>> > > we'll
>>>> > > > need to re-install.
>>>> > > >
>>>> > > > Mike
>>>> > > >
>>>> > > >
>>>> > > > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <
>>>> ottobackwards@gmail.com>
>>>> > > > wrote:
>>>> > > >
>>>> > > > > File
>>>> > > > > "/usr/lib/python2.6/site-packages/resource_management/
>>>> > core/shell.py",
>>>> > > > line
>>>> > > > > 293, in _call
>>>> > > > >
>>>> > > > > raise ExecutionFailed(err_msg, code, out, err)
>>>> > > > >
>>>> > > > > ExecutionFailed: Execution of 'service elasticsearch status'
>>>> returned
>>>> > > 3.
>>>> > > > ●
>>>> > > > > elasticsearch.service - Elasticsearch
>>>> > > > >
>>>> > > > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
>>>> > > > > disabled;
>>>> > > > > vendor preset: disabled)
>>>> > > > >
>>>> > > > > Active: failed (Result: exit-code) since Tue 2017-04-25
>>>> 22:14:58 EDT
>>>> > > > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
>>>> > > > >
>>>> > > > > Docs: http://www.elastic.co
>>>> > > > >
>>>> > > > > Process: 16821 ExecStart=/usr/share/elasticse
>>>> arch/bin/elasticsearch
>>>> > > > > -Des.pidfile=${PID_DIR}/elasticsearch.pid
>>>> > -Des.default.path.home=${ES_
>>>> > > > > HOME}
>>>> > > > > -Des.default.path.logs=${LOG_DIR}
>>>> -Des.default.path.data=${DATA_DIR}
>>>> > > > > -Des.default.path.conf=${CONF_DIR} (code=exited,
>>>> status=1/FAILURE)
>>>> > > > >
>>>> > > > > Process: 16819
>>>> > > > > ExecStartPre=/usr/share/elasticsearch/bin/
>>>> > > elasticsearch-systemd-pre-exec
>>>> > > > > (code=exited, status=0/SUCCESS)
>>>> > > > >
>>>> > > > > Main PID: 16821 (code=exited, status=1/FAILURE)
>>>> > > > >
>>>> > > > >
>>>> > > > >
>>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>>> 04-25%2022:14:58%20EDT
>>>> > >
>>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>>> elasticsearch[16821]:
>>>> > at
>>>> > > > > org.elasticsearch.common.settings.Settings$Builder.
>>>> > > > > loadFromStream(Settings.java:1080)
>>>> > > > >
>>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>>> 04-25%2022:14:58%20EDT
>>>> > >
>>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>>> elasticsearch[16821]:
>>>> > at
>>>> > > > > org.elasticsearch.common.settings.Settings$Builder.
>>>> > > > > loadFromPath(Settings.java:1067)
>>>> > > > >
>>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>>> 04-25%2022:14:58%20EDT
>>>> > >
>>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>>> elasticsearch[16821]:
>>>> > at
>>>> > > > > org.elasticsearch.node.internal.InternalSettingsPreparer.
>>>> > > > > prepareEnvironment(InternalSettingsPreparer.java:88)
>>>> > > > >
>>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>>> 04-25%2022:14:58%20EDT
>>>> > >
>>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>>> elasticsearch[16821]:
>>>> > at
>>>> > > > > org.elasticsearch.bootstrap.Bootstrap.initialSettings(
>>>> > > Bootstrap.java:202)
>>>> > > > >
>>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>>> 04-25%2022:14:58%20EDT
>>>> > >
>>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>>> elasticsearch[16821]:
>>>> > at
>>>> > > > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
>>>> > > > >
>>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>>> 04-25%2022:14:58%20EDT
>>>> > >
>>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>>> elasticsearch[16821]:
>>>> > at
>>>> > > > > org.elasticsearch.bootstrap.Elasticsearch.main(
>>>> > Elasticsearch.java:35)
>>>> > > > >
>>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>>> 04-25%2022:14:58%20EDT
>>>> > >
>>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>>> elasticsearch[16821]:
>>>> > > Refer
>>>> > > > > to
>>>> > > > > the log for complete error details.
>>>> > > > >
>>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>>> 04-25%2022:14:58%20EDT
>>>> > >
>>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
>>>> > > > > elasticsearch.service: main process exited, code=exited,
>>>> > > status=1/FAILURE
>>>> > > > >
>>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>>> 04-25%2022:14:58%20EDT
>>>> > >
>>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
>>>> > > > > elasticsearch.service entered failed state.
>>>> > > > >
>>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>>> 04-25%2022:14:58%20EDT
>>>> > >
>>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
>>>> > > > > elasticsearch.service failed.
>>>> > > > >
>>>> > > > > INFO 2017-04-26 08:55:19,962
>>>> > > > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
>>>> > > > >
>>>> > > > >
>>>> > > > > On April 26, 2017 at 08:27:27, Otto Fowler (
>>>> ottobackwards@gmail.com)
>>>> > > > > wrote:
>>>> > > > >
>>>> > > > > Can you describe what ES issues you are working on?
>>>> > > > > Fixing the repos got everything installed, but my ES components
>>>> don’t
>>>> > > > stay
>>>> > > > > running.
>>>> > > > > I need to harvest the errors.
>>>> > > > >
>>>> > > > >
>>>> > > > >
>>>> > > > > On April 25, 2017 at 16:46:00, Otto Fowler (
>>>> ottobackwards@gmail.com)
>>>> > > > > wrote:
>>>> > > > >
>>>> > > > > Nm. sorry. I fixed it.
>>>> > > > >
>>>> > > > >
>>>> > > > >
>>>> > > > > On April 25, 2017 at 16:42:05, Otto Fowler (
>>>> ottobackwards@gmail.com)
>>>> > > > > wrote:
>>>> > > > >
>>>> > > > > Ok, now I see the repos in the ‘pick version’ screen, but it is
>>>> > erring
>>>> > > on
>>>> > > > > the f://localrepo
>>>> > > > > even though the folder exists, there is no repodata/repomd.xml.
>>>> > > > >
>>>> > > > > What is the command to create a local repo?
>>>> > > > >
>>>> > > > >
>>>> > > > >
>>>> > > > > On April 25, 2017 at 16:05:17, Otto Fowler (
>>>> ottobackwards@gmail.com)
>>>> > > > > wrote:
>>>> > > > >
>>>> > > > > I was going by the HW community page.
>>>> > > > >
>>>> > > > > Ok, Let me try it
>>>> > > > >
>>>> > > > >
>>>> > > > >
>>>> > > > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com
>>>> )
>>>> > > wrote:
>>>> > > > >
>>>> > > > > That would do it. It requires 2.4.2+. I would have sworn I put
>>>> that
>>>> > in
>>>> > > > the
>>>> > > > > README, but I must have only annotated the PR. :(
>>>> > > > >
>>>> > > > > I'll get that in asap.
>>>> > > > >
>>>> > > > > -D...
>>>> > > > >
>>>> > > > >
>>>> > > > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <
>>>> > ottobackwards@gmail.com>
>>>> > > > > wrote:
>>>> > > > >
>>>> > > > > > 2.4.1 as a matter of fact
>>>> > > > > >
>>>> > > > > >
>>>> > > > > > On April 25, 2017 at 15:29:43, David Lyle (
>>>> dlyle65535@gmail.com)
>>>> > > > wrote:
>>>> > > > > >
>>>> > > > > > Any chance they're running Ambari < 2.4.2?
>>>> > > > > >
>>>> > > > > > -D...
>>>> > > > > >
>>>> > > > > >
>>>> > > > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
>>>> > > > > > michael.miklavcic@gmail.com> wrote:
>>>> > > > > >
>>>> > > > > > > Hey Otto, I don't have wizard screenshots in front of me
>>>> right
>>>> > now
>>>> > > to
>>>> > > > > > say
>>>> > > > > > > for sure, but I do see a repoinfo.xml in the mpack. I
>>>> haven't run
>>>> > > > into
>>>> > > > > > > anything like that, but next time I run through the install
>>>> I can
>>>> > > > take
>>>> > > > > > > another look.
>>>> > > > > > >
>>>> > > > > > > https://github.com/apache/incubator-metron/blob/master/
>>>> > > > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
>>>> > > > > > > resources/addon-services/ELAST
>>>> ICSEARCH/2.3.3/repos/repoinfo.xml
>>>> > > > > > >
>>>> > > > > > >
>>>> > > > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
>>>> > > > ottobackwards@gmail.com
>>>> > > > > >
>>>> > > > > > > wrote:
>>>> > > > > > >
>>>> > > > > > > > stderr:
>>>> > > > > > > >
>>>> > > > > > > > Traceback (most recent call last):
>>>> > > > > > > >
>>>> > > > > > > > File "/var/lib/ambari-agent/cache/
>>>> > common-services/ELASTICSEARCH/
>>>> > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in
>>>> <module>
>>>> > > > > > > >
>>>> > > > > > > > Elasticsearch().execute()
>>>> > > > > > > >
>>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>>> ges/resource_management/
>>>> > > > > > > libraries/script/script.py",
>>>> > > > > > > > line 280, in execute
>>>> > > > > > > >
>>>> > > > > > > > method(env)
>>>> > > > > > > >
>>>> > > > > > > > File "/var/lib/ambari-agent/cache/
>>>> > common-services/ELASTICSEARCH/
>>>> > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in
>>>> install
>>>> > > > > > > >
>>>> > > > > > > > self.install_packages(env)
>>>> > > > > > > >
>>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>>> ges/resource_management/
>>>> > > > > > > libraries/script/script.py",
>>>> > > > > > > > line 567, in install_packages
>>>> > > > > > > >
>>>> > > > > > > > retry_count=agent_stack_retry_count)
>>>> > > > > > > >
>>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>>> ges/resource_management/
>>>> > > > > > > core/base.py",
>>>> > > > > > > > line 155, in __init__
>>>> > > > > > > >
>>>> > > > > > > > self.env.run()
>>>> > > > > > > >
>>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>>> ges/resource_management/
>>>> > > > > > > core/environment.py",
>>>> > > > > > > > line 160, in run
>>>> > > > > > > >
>>>> > > > > > > > self.run_action(resource, action)
>>>> > > > > > > >
>>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>>> ges/resource_management/
>>>> > > > > > > core/environment.py",
>>>> > > > > > > > line 124, in run_action
>>>> > > > > > > >
>>>> > > > > > > > provider_action()
>>>> > > > > > > >
>>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>>> ges/resource_management/
>>>> > > > > > > > core/providers/package/__init__.py", line 54, in
>>>> > action_install
>>>> > > > > > > >
>>>> > > > > > > > self.install_package(package_name,
>>>> self.resource.use_repos,
>>>> > > > > > > > self.resource.skip_repos)
>>>> > > > > > > >
>>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>>> ges/resource_management/
>>>> > > > > > > > core/providers/package/yumrpm.py", line 49, in
>>>> install_package
>>>> > > > > > > >
>>>> > > > > > > > self.checked_call_with_retries(cmd, sudo=True,
>>>> > > > > > > > logoutput=self.get_logoutput())
>>>> > > > > > > >
>>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>>> ges/resource_management/
>>>> > > > > > > > core/providers/package/__init__.py", line 83, in
>>>> > > > > > > checked_call_with_retries
>>>> > > > > > > >
>>>> > > > > > > > return self._call_with_retries(cmd, is_checked=True,
>>>> **kwargs)
>>>> > > > > > > >
>>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>>> ges/resource_management/
>>>> > > > > > > > core/providers/package/__init__.py", line 91, in
>>>> > > _call_with_retries
>>>> > > > > > > >
>>>> > > > > > > > code, out = func(cmd, **kwargs)
>>>> > > > > > > >
>>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>>> ges/resource_management/
>>>> > > > > > > core/shell.py",
>>>> > > > > > > > line 71, in inner
>>>> > > > > > > >
>>>> > > > > > > > result = function(command, **kwargs)
>>>> > > > > > > >
>>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>>> ges/resource_management/
>>>> > > > > > > core/shell.py",
>>>> > > > > > > > line 93, in checked_call
>>>> > > > > > > >
>>>> > > > > > > > tries=tries, try_sleep=try_sleep)
>>>> > > > > > > >
>>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>>> ges/resource_management/
>>>> > > > > > > core/shell.py",
>>>> > > > > > > > line 141, in _call_wrapper
>>>> > > > > > > >
>>>> > > > > > > > result = _call(command, **kwargs_copy)
>>>> > > > > > > >
>>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>>> ges/resource_management/
>>>> > > > > > > core/shell.py",
>>>> > > > > > > > line 294, in _call
>>>> > > > > > > >
>>>> > > > > > > > raise Fail(err_msg)
>>>> > > > > > > >
>>>> > > > > > > > resource_management.core.exceptions.Fail: Execution of
>>>> > > > '/usr/bin/yum
>>>> > > > > > -d
>>>> > > > > > > 0
>>>> > > > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error:
>>>> Nothing
>>>> > > to
>>>> > > > do
>>>> > > > > > > >
>>>> > > > > > > > stdout:
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,669
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> Using
>>>> > > > hadoop
>>>> > > > > > > conf
>>>> > > > > > > > dir: /usr/hdp/current/hadoop-client/conf
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,671
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> > > > > > Group['metron']
>>>> > > > > > > {}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,673
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> Adding
>>>> > > > group
>>>> > > > > > > > Group['metron']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,740
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> > > > > Group['livy']
>>>> > > > > > {}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,741
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> Adding
>>>> > > > group
>>>> > > > > > > > Group['livy']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,774
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> > > > > > > > Group['elasticsearch'] {}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,775
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> Adding
>>>> > > > group
>>>> > > > > > > > Group['elasticsearch']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,812
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> > > > > > Group['spark']
>>>> > > > > > > {}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,812
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> Adding
>>>> > > > group
>>>> > > > > > > > Group['spark']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,848
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> > > > > > > Group['zeppelin']
>>>> > > > > > > > {}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,849
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> Adding
>>>> > > > group
>>>> > > > > > > > Group['zeppelin']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,884
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> > > > > > Group['hadoop']
>>>> > > > > > > {}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,885
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> Adding
>>>> > > > group
>>>> > > > > > > > Group['hadoop']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,921
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> > > > > > Group['kibana']
>>>> > > > > > > {}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,922
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> Adding
>>>> > > > group
>>>> > > > > > > > Group['kibana']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,959
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> > > > > > Group['users']
>>>> > > > > > > {}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,960
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> > > > User['hive']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:48,961
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['hive']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,227
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> > > > > User['storm']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,227
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['storm']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,388
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> > > > > > > User['zookeeper']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,388
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['zookeeper']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,518
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> > > > > > > > User['infra-solr'] {'gid': 'hadoop',
>>>> 'fetch_nonlocal_groups':
>>>> > > True,
>>>> > > > > > > > 'groups': [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,518
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['infra-solr']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,578
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> > > > User['ams']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,578
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['ams']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,625
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> > > > User['tez']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > [u'users']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,626
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['tez']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,660
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> > > > > > User['zeppelin']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,660
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['zeppelin']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,705
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> > > > > > User['metron']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,705
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['metron']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,760
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> > > > User['livy']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,760
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['livy']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,803
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> > > > > > > > User['elasticsearch'] {'gid': 'hadoop',
>>>> > 'fetch_nonlocal_groups':
>>>> > > > > True,
>>>> > > > > > > > 'groups': [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,804
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['elasticsearch']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,837
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> > > > > User['spark']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,837
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['spark']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,885
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> > > > > > > User['ambari-qa']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > [u'users']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,885
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['ambari-qa']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,928
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> > > > > User['flume']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,928
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['flume']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,972
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> > > > > User['kafka']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:49,973
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['kafka']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,025
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > User['hdfs']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,026
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['hdfs']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,075
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > User['yarn']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,075
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['yarn']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,125
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > > > User['kibana']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,126
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['kibana']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,175
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > > > User['mapred']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,176
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['mapred']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,223
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > > User['hbase']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,224
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['hbase']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,271
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > User['hcat']
>>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>>> > > > > > [u'hadoop']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,272
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> Adding
>>>> > > > user
>>>> > > > > > > > User['hcat']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,320
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh']
>>>> {'content':
>>>> > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,333
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > Writing
>>>> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it
>>>> > > doesn't
>>>> > > > > > exist
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,334
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > Changing
>>>> > > > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh
>>>> from 644
>>>> > > to
>>>> > > > > 555
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,335
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
>>>> > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
>>>> > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa']
>>>> {'not_if':
>>>> > > '(test
>>>> > > > > > $(id
>>>> > > > > > > -u
>>>> > > > > > > > ambari-qa) -gt 1000) || (false)'}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,364
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > Skipping
>>>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
>>>> > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
>>>> > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to
>>>> not_if
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,365
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
>>>> > > 'create_parents':
>>>> > > > > > True,
>>>> > > > > > > > 'mode': 0775, 'cd_access': 'a'}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,366
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > Creating
>>>> > > > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't
>>>> exist.
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,366
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > Changing
>>>> > > > > > owner
>>>> > > > > > > > for /tmp/hbase-hbase from 0 to hbase
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,366
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > Changing
>>>> > > > > > > > permission for /tmp/hbase-hbase from 755 to 775
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,368
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh']
>>>> {'content':
>>>> > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,369
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
>>>> > > > > > > >
>>>> > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
>>>> > > hbase-hbase']
>>>> > > > > >
>>>> > > > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,393
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > Skipping
>>>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
>>>> > > > > > > >
>>>> > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
>>>> > > hbase-hbase']
>>>> > > > > >
>>>> > > > > > > > due to not_if
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,394
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > > Group['hdfs']
>>>> > > > > > {}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,395
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> Adding
>>>> > > > group
>>>> > > > > > > > Group['hdfs']
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,430
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > User['hdfs']
>>>> > > > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop',
>>>> u'hdfs']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,431
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > Modifying
>>>> > > > > > user
>>>> > > > > > > > hdfs
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,474
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> FS
>>>> > Type:
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,474
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > > > > > Directory['/etc/hadoop'] {'mode': 0755}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,474
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > Creating
>>>> > > > > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,475
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > > > > > Directory['/var/lib/ambari-age
>>>> nt/tmp/hadoop_java_io_tmpdir']
>>>> > > > > > {'owner':
>>>> > > > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,476
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > Creating
>>>> > > > > > > > directory Directory['/var/lib/ambari-age
>>>> nt/tmp/hadoop_java_io_
>>>> > > > > tmpdir']
>>>> > > > > >
>>>> > > > > > > > since it doesn't exist.
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,476
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > Changing
>>>> > > > > > owner
>>>> > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
>>>> 0 to
>>>> > > hdfs
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,476
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > Changing
>>>> > > > > > group
>>>> > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
>>>> 0 to
>>>> > > > hadoop
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,477
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > Changing
>>>> > > > > > > > permission for /var/lib/ambari-agent/tmp/hado
>>>> op_java_io_tmpdir
>>>> > > from
>>>> > > > > > 755
>>>> > > > > > > > to 1777
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,477
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
>>>> > > > > > > > {'create_parents': True}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,478
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > Creating
>>>> > > > > > > > directory Directory['/var/lib/ambari-
>>>> > > agent/tmp/AMBARI-artifacts/']
>>>> > > > > > since
>>>> > > > > > > > it doesn't exist.
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,479
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > > > > > File['/var/lib/ambari-agent/tm
>>>> p/jdk-8u77-linux-x64.tar.gz']
>>>> > > > > > {'content':
>>>> > > > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
>>>> > > > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
>>>> > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:50,504
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>>> > > > Downloading
>>>> > > > > > the
>>>> > > > > > > > file from http://css-fx-met-poc-82:8080/
>>>> resources//jdk-8u77-
>>>> > > > > > > > linux-x64.tar.gz
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:53,793
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>>>> > > > > > > > Directory['/usr/jdk64'] {}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:53,793
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>>>> > Creating
>>>> > > > > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:53,794
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>>>> > > > > > > Execute[('chmod',
>>>> > > > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:12:53,824
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>>>> > > > Execute['cd
>>>> > > > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
>>>> > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
>>>> > > > > ambari-sudo.sh
>>>> > > > > > cp
>>>> > > > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/*
>>>> /usr/jdk64'] {}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:00,439
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>>> > > > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
>>>> > {'action':
>>>> > > > > > > > ['delete']}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:00,440
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>>> > Removing
>>>> > > > > > > > directory Directory['/var/lib/ambari-
>>>> > agent/tmp/jdk_tmp_1BRMsS']
>>>> > > and
>>>> > > > > > all
>>>> > > > > > > > its content
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:00,620
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>>> > > > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
>>>> > > 'cd_access':
>>>> > > > > > 'a'}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:00,621
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>>> > > > > > > Execute[('chmod',
>>>> > > > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:00,682
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>>> > > > Initializing
>>>> > > > > > 2
>>>> > > > > > > > repositories
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:00,683
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>>> > > > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
>>>> > > > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0',
>>>> 'action':
>>>> > > > > > ['create'],
>>>> > > > > > > > 'components': [u'HDP', 'main'], 'repo_template':
>>>> > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
>>>> > > > > > > > %}mirrorlist={{mirror_list}}{% else
>>>> %}baseurl={{base_url}}{%
>>>> > > endif
>>>> > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
>>>> 'HDP',
>>>> > > > > > > > 'mirror_list': None}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:00,709
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>>> > > > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content':
>>>> > > InlineTemplate(...)}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:00,711
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>>> > Writing
>>>> > > > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't
>>>> exist
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:00,711
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>>> > > > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
>>>> > > > http://public-repo-1.
>>>> > > >
>>>> > > > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7',
>>>> 'action':
>>>> > > > > > ['create'],
>>>> > > > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
>>>> > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
>>>> > > > > > > > %}mirrorlist={{mirror_list}}{% else
>>>> %}baseurl={{base_url}}{%
>>>> > > endif
>>>> > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
>>>> > > > 'HDP-UTILS',
>>>> > > > > > > > 'mirror_list': None}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:00,717
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>>> > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
>>>> > > > > > InlineTemplate(...)}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:00,717
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>>> > Writing
>>>> > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it
>>>> doesn't
>>>> > exist
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:00,718
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>>> > > > > > Package['unzip']
>>>> > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:00,846
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>>> > > Installing
>>>> > > > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:03,413
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>>>> > > > > > Package['curl']
>>>> > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:03,429
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>>>> > Skipping
>>>> > > > > > > > installation of existing package curl
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:03,430
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>>>> > > > > > > > Package['hdp-select'] {'retry_on_repo_unavailability':
>>>> False,
>>>> > > > > > > > 'retry_count': 5}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:03,451
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>>>> > > Installing
>>>> > > > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
>>>> > > hdp-select')
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:04,948
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
>>>> > Install
>>>> > > ES
>>>> > > > > > Data
>>>> > > > > > > > Node
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:04,952
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
>>>> > > > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_
>>>> > unavailability':
>>>> > > > > > False,
>>>> > > > > > > > 'retry_count': 5}
>>>> > > > > > > >
>>>> > > > > > > > 2017-04-25 14:13:05,086
>>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> -
>>>> > > Installing
>>>> > > > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y
>>>> install
>>>> > > > > > > > elasticsearch-2.3.3')
>>>> > > > > > > >
>>>> > > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > Command failed after 1 tries
>>>> > > > > > > >
>>>> > > > > > > >
>>>> > > > > > > >
>>>> > > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
>>>> > > > ottobackwards@gmail.com)
>>>> > > >
>>>> > > > > > > > wrote:
>>>> > > > > > > >
>>>> > > > > > > > Yes, I’m going to run through again and get the logs. The
>>>> basic
>>>> > > > issue
>>>> > > > > > > is,
>>>> > > > > > > > no repo tab -> if I should have one,
>>>> > > > > > > > and the ES yum command failing with nothing to do, which
>>>> stops
>>>> > > > > > deployment
>>>> > > > > > > > on all the nodes.
>>>> > > > > > > >
>>>> > > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
>>>> > > > > > > > michael.miklavcic@gmail.com) wrote:
>>>> > > > > > > >
>>>> > > > > > > > I've been doing an internal hwx deployment (HCP) recently
>>>> for
>>>> > > > testing
>>>> > > > > > > > purposes. I have some ES tweaks to work out, but other
>>>> than
>>>> > that
>>>> > > it
>>>> > > > > > has
>>>> > > > > > > > been running smoothly. Can you share any log info or
>>>> > screenshots?
>>>> > > > > > > >
>>>> > > > > > > > Mike
>>>> > > > > > > >
>>>> > > > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <
>>>> > ottobackwards@gmail.com
>>>> > > >
>>>> > > > > > wrote:
>>>> > > > > > > >
>>>> > > > > > > > > Hi,
>>>> > > > > > > > > Should I see the repo tab in the ambari wizard for
>>>> metron at
>>>> > > this
>>>> > > > > > > point?
>>>> > > > > > > > > Asking for a friend who can’t get the wizard to work :)
>>>> > > > > > > > >
>>>> > > > > > > >
>>>> > > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > > --
>>>> > >
>>>> > > Jon
>>>> > >
>>>> > >
>>>> >
>>>>
>>>>
>>>
>>
>
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
Thanks Nick. I don’t think we catch exceptions at all from the execute
calls. I wonder what the best practice is?
I can’t seem to find the ambari mpack programming documentation ;)
On April 26, 2017 at 16:50:18, Nick Allen (nick@nickallen.org) wrote:
I can create the JIRA. I capture the logs.
On Wed, Apr 26, 2017 at 4:48 PM, Nick Allen <ni...@nickallen.org> wrote:
> Yes, Otto. I just experienced that myself. It is a bug that we should
> create a JIRA for.
>
> I was able to work around it by just using "start" instead of "restart".
>
> On Wed, Apr 26, 2017 at 3:41 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
>> Ok, I did ifconfig, i mean ip a and got the right values
>> that and the brackets and it looks like es is up.
>>
>> Now my next problem is prob. a bug.
>>
>> Ambari thinks it needs to restart ‘metron’.
>> the enrichment restart crashes because we don’t catch the storm exception
>> that you are stopping a topology that was not running.
>>
>> The question is - is the state wrong _and_ the exception catch or not.
>>
>> I am guessing I should start a Jira for that one?
>>
>>
>>
>> On April 26, 2017 at 15:31:05, Otto Fowler (ottobackwards@gmail.com)
>> wrote:
>>
>> So now I’m getting errors for ETH0
>> I think that is one of the things that has to change on centos7?
>> Can you check what you have?
>>
>>
>>
>> On April 26, 2017 at 15:20:18, Nick Allen (nick@nickallen.org) wrote:
>>
>> Yes, I am also running on CentOS 7.
>>
>> On Wed, Apr 26, 2017 at 3:19 PM, Otto Fowler <ot...@gmail.com>
>> wrote:
>>
>>> Would i have to change that on Centos 7?
>>>
>>>
>>> On April 26, 2017 at 15:12:38, Nick Allen (nick@nickallen.org) wrote:
>>>
>>> Check on the `network_hosts` setting. I just ran into the same issue. I
>>> had to add brackets around it otherwise the file was not valid YAML and
>>> it
>>> would crash trying to load the config file.
>>>
>>> network.host: ["_lo:ipv4_","_eth0:ipv4_"]
>>>
>>> On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler <ot...@gmail.com>
>>> wrote:
>>>
>>> > I think it is literally crashing trying to load the config file
>>> >
>>> >
>>> >
>>> > On April 26, 2017 at 15:00:25, Michael Miklavcic (
>>> > michael.miklavcic@gmail.com) wrote:
>>> >
>>> > A couple more things you should check, given this setup:
>>> >
>>> > - 1 master node (not as data node)
>>> > - 2 data nodes
>>> >
>>> > I'd use something like this:
>>> >
>>> > - gateway_recover_after_data_nodes=1 or 2
>>> > - index_number_of_replicas=0 or 1 (2 data nodes, with 2 total copies of
>>> > the index - if you set this value to 2, it would need 1 more data node
>>> > to
>>> > attain 1 orig index + 2 replicas)
>>> >
>>> >
>>> >
>>> > On Wed, Apr 26, 2017 at 12:02 PM, Otto Fowler <ottobackwards@gmail.com
>>> >
>>> > wrote:
>>> >
>>> > > Actually never mind. I am not sure how to read the exception. Is it
>>> > > failing to read the configuration?
>>> > >
>>> > >
>>> > > On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwards@gmail.com)
>>> > > wrote:
>>> > >
>>> > > So the elasticsearch service is disabled. That is the issue. When ES
>>> > > calls get status ( service elasticsearch status ) it fails.
>>> > > I don’t know systemd too well, but does that mean we didn’t do the
>>> equiv
>>> > > of chkconfig on?
>>> > >
>>> > >
>>> > >
>>> > > On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com)
>>> > > wrote:
>>> > >
>>> > > Getting the same error in ES as before from what I can see
>>> > >
>>> > >
>>> > > On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com)
>>> > > wrote:
>>> > >
>>> > > I still can’t keep ES running, it dies.
>>> > > Also, I’m getting exceptions from enrichment trying to restart when
>>> the
>>> > > topology wasn’t running.
>>> > >
>>> > > Is there some coordination in this work? Should I just log jiras?
>>> > > If someone is already sussing out ES, we can sync up
>>> > >
>>> > >
>>> > >
>>> > > On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com)
>>> > wrote:
>>> > >
>>> > > I'm also interested to know why that's important at such a small
>>> scale.
>>> > >
>>> > > Jon
>>> > >
>>> > > On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com>
>>> > > wrote:
>>> > >
>>> > > > I am following
>>> > > >
>>> > > > https://community.hortonworks.com/articles/60805/deploying-
>>> > > a-fresh-metron-cluster-using-ambari-serv.html
>>> > > > I DO have master and data node together.
>>> > > > Why is that a problem?
>>> > > >
>>> > > > I will try again with master and data node separate.
>>> > > >
>>> > > >
>>> > > > On April 26, 2017 at 10:41:59, Michael Miklavcic (
>>> > > > michael.miklavcic@gmail.com) wrote:
>>> > > >
>>> > > > Hey Otto,
>>> > > >
>>> > > > How do you have the ES nodes configured? For a base install I would
>>> > > setup 1
>>> > > > master (NOT as data node) and 2 data nodes (NOT on the same node
>>> as the
>>> > > > master). This is the install configuration I got working. You can
>>> also
>>> > > > modify some configuration properties around master node as data
>>> node,
>>> > > index
>>> > > > replicas, and gateway recovery to get it working differently, but
>>> this
>>> > is
>>> > > > what will work OOTB with the default config settings from the
>>> mpack. If
>>> > > > you've already setup a master node and a data node on the same
>>> host,
>>> > > we'll
>>> > > > need to re-install.
>>> > > >
>>> > > > Mike
>>> > > >
>>> > > >
>>> > > > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <
>>> ottobackwards@gmail.com>
>>> > > > wrote:
>>> > > >
>>> > > > > File
>>> > > > > "/usr/lib/python2.6/site-packages/resource_management/
>>> > core/shell.py",
>>> > > > line
>>> > > > > 293, in _call
>>> > > > >
>>> > > > > raise ExecutionFailed(err_msg, code, out, err)
>>> > > > >
>>> > > > > ExecutionFailed: Execution of 'service elasticsearch status'
>>> returned
>>> > > 3.
>>> > > > ●
>>> > > > > elasticsearch.service - Elasticsearch
>>> > > > >
>>> > > > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
>>> > > > > disabled;
>>> > > > > vendor preset: disabled)
>>> > > > >
>>> > > > > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58
>>> EDT
>>> > > > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
>>> > > > >
>>> > > > > Docs: http://www.elastic.co
>>> > > > >
>>> > > > > Process: 16821 ExecStart=/usr/share/elasticse
>>> arch/bin/elasticsearch
>>> > > > > -Des.pidfile=${PID_DIR}/elasticsearch.pid
>>> > -Des.default.path.home=${ES_
>>> > > > > HOME}
>>> > > > > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_
>>> DIR}
>>> > > > > -Des.default.path.conf=${CONF_DIR} (code=exited,
>>> status=1/FAILURE)
>>> > > > >
>>> > > > > Process: 16819
>>> > > > > ExecStartPre=/usr/share/elasticsearch/bin/
>>> > > elasticsearch-systemd-pre-exec
>>> > > > > (code=exited, status=0/SUCCESS)
>>> > > > >
>>> > > > > Main PID: 16821 (code=exited, status=1/FAILURE)
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>> elasticsearch[16821]:
>>> > at
>>> > > > > org.elasticsearch.common.settings.Settings$Builder.
>>> > > > > loadFromStream(Settings.java:1080)
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>> elasticsearch[16821]:
>>> > at
>>> > > > > org.elasticsearch.common.settings.Settings$Builder.
>>> > > > > loadFromPath(Settings.java:1067)
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>> elasticsearch[16821]:
>>> > at
>>> > > > > org.elasticsearch.node.internal.InternalSettingsPreparer.
>>> > > > > prepareEnvironment(InternalSettingsPreparer.java:88)
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>> elasticsearch[16821]:
>>> > at
>>> > > > > org.elasticsearch.bootstrap.Bootstrap.initialSettings(
>>> > > Bootstrap.java:202)
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>> elasticsearch[16821]:
>>> > at
>>> > > > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>> elasticsearch[16821]:
>>> > at
>>> > > > > org.elasticsearch.bootstrap.Elasticsearch.main(
>>> > Elasticsearch.java:35)
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>> elasticsearch[16821]:
>>> > > Refer
>>> > > > > to
>>> > > > > the log for complete error details.
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
>>> > > > > elasticsearch.service: main process exited, code=exited,
>>> > > status=1/FAILURE
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
>>> > > > > elasticsearch.service entered failed state.
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
>>> > > > > elasticsearch.service failed.
>>> > > > >
>>> > > > > INFO 2017-04-26 08:55:19,962
>>> > > > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
>>> > > > >
>>> > > > >
>>> > > > > On April 26, 2017 at 08:27:27, Otto Fowler (
>>> ottobackwards@gmail.com)
>>> > > > > wrote:
>>> > > > >
>>> > > > > Can you describe what ES issues you are working on?
>>> > > > > Fixing the repos got everything installed, but my ES components
>>> don’t
>>> > > > stay
>>> > > > > running.
>>> > > > > I need to harvest the errors.
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > On April 25, 2017 at 16:46:00, Otto Fowler (
>>> ottobackwards@gmail.com)
>>> > > > > wrote:
>>> > > > >
>>> > > > > Nm. sorry. I fixed it.
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > On April 25, 2017 at 16:42:05, Otto Fowler (
>>> ottobackwards@gmail.com)
>>> > > > > wrote:
>>> > > > >
>>> > > > > Ok, now I see the repos in the ‘pick version’ screen, but it is
>>> > erring
>>> > > on
>>> > > > > the f://localrepo
>>> > > > > even though the folder exists, there is no repodata/repomd.xml.
>>> > > > >
>>> > > > > What is the command to create a local repo?
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > On April 25, 2017 at 16:05:17, Otto Fowler (
>>> ottobackwards@gmail.com)
>>> > > > > wrote:
>>> > > > >
>>> > > > > I was going by the HW community page.
>>> > > > >
>>> > > > > Ok, Let me try it
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com)
>>> > > wrote:
>>> > > > >
>>> > > > > That would do it. It requires 2.4.2+. I would have sworn I put
>>> that
>>> > in
>>> > > > the
>>> > > > > README, but I must have only annotated the PR. :(
>>> > > > >
>>> > > > > I'll get that in asap.
>>> > > > >
>>> > > > > -D...
>>> > > > >
>>> > > > >
>>> > > > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <
>>> > ottobackwards@gmail.com>
>>> > > > > wrote:
>>> > > > >
>>> > > > > > 2.4.1 as a matter of fact
>>> > > > > >
>>> > > > > >
>>> > > > > > On April 25, 2017 at 15:29:43, David Lyle (
>>> dlyle65535@gmail.com)
>>> > > > wrote:
>>> > > > > >
>>> > > > > > Any chance they're running Ambari < 2.4.2?
>>> > > > > >
>>> > > > > > -D...
>>> > > > > >
>>> > > > > >
>>> > > > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
>>> > > > > > michael.miklavcic@gmail.com> wrote:
>>> > > > > >
>>> > > > > > > Hey Otto, I don't have wizard screenshots in front of me
>>> right
>>> > now
>>> > > to
>>> > > > > > say
>>> > > > > > > for sure, but I do see a repoinfo.xml in the mpack. I
>>> haven't run
>>> > > > into
>>> > > > > > > anything like that, but next time I run through the install
>>> I can
>>> > > > take
>>> > > > > > > another look.
>>> > > > > > >
>>> > > > > > > https://github.com/apache/incubator-metron/blob/master/
>>> > > > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
>>> > > > > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.
>>> xml
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
>>> > > > ottobackwards@gmail.com
>>> > > > > >
>>> > > > > > > wrote:
>>> > > > > > >
>>> > > > > > > > stderr:
>>> > > > > > > >
>>> > > > > > > > Traceback (most recent call last):
>>> > > > > > > >
>>> > > > > > > > File "/var/lib/ambari-agent/cache/
>>> > common-services/ELASTICSEARCH/
>>> > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in
>>> <module>
>>> > > > > > > >
>>> > > > > > > > Elasticsearch().execute()
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > libraries/script/script.py",
>>> > > > > > > > line 280, in execute
>>> > > > > > > >
>>> > > > > > > > method(env)
>>> > > > > > > >
>>> > > > > > > > File "/var/lib/ambari-agent/cache/
>>> > common-services/ELASTICSEARCH/
>>> > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in
>>> install
>>> > > > > > > >
>>> > > > > > > > self.install_packages(env)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > libraries/script/script.py",
>>> > > > > > > > line 567, in install_packages
>>> > > > > > > >
>>> > > > > > > > retry_count=agent_stack_retry_count)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > core/base.py",
>>> > > > > > > > line 155, in __init__
>>> > > > > > > >
>>> > > > > > > > self.env.run()
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > core/environment.py",
>>> > > > > > > > line 160, in run
>>> > > > > > > >
>>> > > > > > > > self.run_action(resource, action)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > core/environment.py",
>>> > > > > > > > line 124, in run_action
>>> > > > > > > >
>>> > > > > > > > provider_action()
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > > core/providers/package/__init__.py", line 54, in
>>> > action_install
>>> > > > > > > >
>>> > > > > > > > self.install_package(package_name,
>>> self.resource.use_repos,
>>> > > > > > > > self.resource.skip_repos)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > > core/providers/package/yumrpm.py", line 49, in
>>> install_package
>>> > > > > > > >
>>> > > > > > > > self.checked_call_with_retries(cmd, sudo=True,
>>> > > > > > > > logoutput=self.get_logoutput())
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > > core/providers/package/__init__.py", line 83, in
>>> > > > > > > checked_call_with_retries
>>> > > > > > > >
>>> > > > > > > > return self._call_with_retries(cmd, is_checked=True,
>>> **kwargs)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > > core/providers/package/__init__.py", line 91, in
>>> > > _call_with_retries
>>> > > > > > > >
>>> > > > > > > > code, out = func(cmd, **kwargs)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > core/shell.py",
>>> > > > > > > > line 71, in inner
>>> > > > > > > >
>>> > > > > > > > result = function(command, **kwargs)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > core/shell.py",
>>> > > > > > > > line 93, in checked_call
>>> > > > > > > >
>>> > > > > > > > tries=tries, try_sleep=try_sleep)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > core/shell.py",
>>> > > > > > > > line 141, in _call_wrapper
>>> > > > > > > >
>>> > > > > > > > result = _call(command, **kwargs_copy)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > core/shell.py",
>>> > > > > > > > line 294, in _call
>>> > > > > > > >
>>> > > > > > > > raise Fail(err_msg)
>>> > > > > > > >
>>> > > > > > > > resource_management.core.exceptions.Fail: Execution of
>>> > > > '/usr/bin/yum
>>> > > > > > -d
>>> > > > > > > 0
>>> > > > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error:
>>> Nothing
>>> > > to
>>> > > > do
>>> > > > > > > >
>>> > > > > > > > stdout:
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,669
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Using
>>> > > > hadoop
>>> > > > > > > conf
>>> > > > > > > > dir: /usr/hdp/current/hadoop-client/conf
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,671
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > > Group['metron']
>>> > > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,673
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['metron']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,740
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > Group['livy']
>>> > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,741
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['livy']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,774
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > > > > Group['elasticsearch'] {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,775
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['elasticsearch']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,812
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > > Group['spark']
>>> > > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,812
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['spark']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,848
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > > > Group['zeppelin']
>>> > > > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,849
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['zeppelin']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,884
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > > Group['hadoop']
>>> > > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,885
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['hadoop']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,921
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > > Group['kibana']
>>> > > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,922
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['kibana']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,959
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > > Group['users']
>>> > > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,960
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > User['hive']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,961
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['hive']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,227
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > User['storm']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,227
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['storm']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,388
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > > > User['zookeeper']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,388
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['zookeeper']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,518
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > > > > User['infra-solr'] {'gid': 'hadoop',
>>> 'fetch_nonlocal_groups':
>>> > > True,
>>> > > > > > > > 'groups': [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,518
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['infra-solr']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,578
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > User['ams']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,578
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['ams']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,625
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > User['tez']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > [u'users']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,626
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['tez']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,660
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > > User['zeppelin']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,660
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['zeppelin']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,705
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > > User['metron']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,705
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['metron']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,760
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > User['livy']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,760
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['livy']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,803
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > > > > User['elasticsearch'] {'gid': 'hadoop',
>>> > 'fetch_nonlocal_groups':
>>> > > > > True,
>>> > > > > > > > 'groups': [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,804
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['elasticsearch']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,837
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > User['spark']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,837
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['spark']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,885
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > > > User['ambari-qa']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > [u'users']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,885
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['ambari-qa']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,928
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > User['flume']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,928
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['flume']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,972
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > User['kafka']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,973
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['kafka']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,025
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > User['hdfs']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,026
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['hdfs']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,075
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > User['yarn']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,075
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['yarn']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,125
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > User['kibana']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,126
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['kibana']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,175
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > User['mapred']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,176
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['mapred']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,223
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > User['hbase']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,224
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['hbase']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,271
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > User['hcat']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,272
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['hcat']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,320
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
>>> > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,333
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Writing
>>> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it
>>> > > doesn't
>>> > > > > > exist
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,334
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Changing
>>> > > > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh
>>> from 644
>>> > > to
>>> > > > > 555
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,335
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
>>> > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
>>> > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if':
>>> > > '(test
>>> > > > > > $(id
>>> > > > > > > -u
>>> > > > > > > > ambari-qa) -gt 1000) || (false)'}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,364
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Skipping
>>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
>>> > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
>>> > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to
>>> not_if
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,365
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
>>> > > 'create_parents':
>>> > > > > > True,
>>> > > > > > > > 'mode': 0775, 'cd_access': 'a'}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,366
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Creating
>>> > > > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't
>>> exist.
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,366
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Changing
>>> > > > > > owner
>>> > > > > > > > for /tmp/hbase-hbase from 0 to hbase
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,366
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Changing
>>> > > > > > > > permission for /tmp/hbase-hbase from 755 to 775
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,368
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
>>> > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,369
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
>>> > > > > > > >
>>> > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
>>> > > hbase-hbase']
>>> > > > > >
>>> > > > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,393
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Skipping
>>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
>>> > > > > > > >
>>> > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
>>> > > hbase-hbase']
>>> > > > > >
>>> > > > > > > > due to not_if
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,394
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > Group['hdfs']
>>> > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,395
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['hdfs']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,430
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > User['hdfs']
>>> > > > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop',
>>> u'hdfs']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,431
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > Modifying
>>> > > > > > user
>>> > > > > > > > hdfs
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,474
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS
>>> > Type:
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,474
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > Directory['/etc/hadoop'] {'mode': 0755}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,474
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Creating
>>> > > > > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,475
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > Directory['/var/lib/ambari-age
>>> nt/tmp/hadoop_java_io_tmpdir']
>>> > > > > > {'owner':
>>> > > > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,476
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Creating
>>> > > > > > > > directory Directory['/var/lib/ambari-age
>>> nt/tmp/hadoop_java_io_
>>> > > > > tmpdir']
>>> > > > > >
>>> > > > > > > > since it doesn't exist.
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,476
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Changing
>>> > > > > > owner
>>> > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
>>> 0 to
>>> > > hdfs
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,476
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Changing
>>> > > > > > group
>>> > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
>>> 0 to
>>> > > > hadoop
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,477
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Changing
>>> > > > > > > > permission for /var/lib/ambari-agent/tmp/hado
>>> op_java_io_tmpdir
>>> > > from
>>> > > > > > 755
>>> > > > > > > > to 1777
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,477
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
>>> > > > > > > > {'create_parents': True}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,478
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Creating
>>> > > > > > > > directory Directory['/var/lib/ambari-
>>> > > agent/tmp/AMBARI-artifacts/']
>>> > > > > > since
>>> > > > > > > > it doesn't exist.
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,479
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > File['/var/lib/ambari-agent/tm
>>> p/jdk-8u77-linux-x64.tar.gz']
>>> > > > > > {'content':
>>> > > > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
>>> > > > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
>>> > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,504
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > Downloading
>>> > > > > > the
>>> > > > > > > > file from http://css-fx-met-poc-82:8080/
>>> resources//jdk-8u77-
>>> > > > > > > > linux-x64.tar.gz
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:53,793
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>>> > > > > > > > Directory['/usr/jdk64'] {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:53,793
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>>> > Creating
>>> > > > > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:53,794
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>>> > > > > > > Execute[('chmod',
>>> > > > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:53,824
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>>> > > > Execute['cd
>>> > > > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
>>> > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
>>> > > > > ambari-sudo.sh
>>> > > > > > cp
>>> > > > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/*
>>> /usr/jdk64'] {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,439
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
>>> > {'action':
>>> > > > > > > > ['delete']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,440
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > Removing
>>> > > > > > > > directory Directory['/var/lib/ambari-
>>> > agent/tmp/jdk_tmp_1BRMsS']
>>> > > and
>>> > > > > > all
>>> > > > > > > > its content
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,620
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
>>> > > 'cd_access':
>>> > > > > > 'a'}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,621
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > > Execute[('chmod',
>>> > > > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,682
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > Initializing
>>> > > > > > 2
>>> > > > > > > > repositories
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,683
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
>>> > > > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0',
>>> 'action':
>>> > > > > > ['create'],
>>> > > > > > > > 'components': [u'HDP', 'main'], 'repo_template':
>>> > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
>>> > > > > > > > %}mirrorlist={{mirror_list}}{% else
>>> %}baseurl={{base_url}}{%
>>> > > endif
>>> > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
>>> 'HDP',
>>> > > > > > > > 'mirror_list': None}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,709
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content':
>>> > > InlineTemplate(...)}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,711
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > Writing
>>> > > > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,711
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
>>> > > > http://public-repo-1.
>>> > > >
>>> > > > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7',
>>> 'action':
>>> > > > > > ['create'],
>>> > > > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
>>> > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
>>> > > > > > > > %}mirrorlist={{mirror_list}}{% else
>>> %}baseurl={{base_url}}{%
>>> > > endif
>>> > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
>>> > > > 'HDP-UTILS',
>>> > > > > > > > 'mirror_list': None}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,717
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
>>> > > > > > InlineTemplate(...)}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,717
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > Writing
>>> > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't
>>> > exist
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,718
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > Package['unzip']
>>> > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,846
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > Installing
>>> > > > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:03,413
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>>> > > > > > Package['curl']
>>> > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:03,429
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>>> > Skipping
>>> > > > > > > > installation of existing package curl
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:03,430
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>>> > > > > > > > Package['hdp-select'] {'retry_on_repo_unavailability':
>>> False,
>>> > > > > > > > 'retry_count': 5}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:03,451
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>>> > > Installing
>>> > > > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
>>> > > hdp-select')
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:04,948
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
>>> > Install
>>> > > ES
>>> > > > > > Data
>>> > > > > > > > Node
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:04,952
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
>>> > > > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_
>>> > unavailability':
>>> > > > > > False,
>>> > > > > > > > 'retry_count': 5}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:05,086
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> -
>>> > > Installing
>>> > > > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y
>>> install
>>> > > > > > > > elasticsearch-2.3.3')
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > Command failed after 1 tries
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
>>> > > > ottobackwards@gmail.com)
>>> > > >
>>> > > > > > > > wrote:
>>> > > > > > > >
>>> > > > > > > > Yes, I’m going to run through again and get the logs. The
>>> basic
>>> > > > issue
>>> > > > > > > is,
>>> > > > > > > > no repo tab -> if I should have one,
>>> > > > > > > > and the ES yum command failing with nothing to do, which
>>> stops
>>> > > > > > deployment
>>> > > > > > > > on all the nodes.
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
>>> > > > > > > > michael.miklavcic@gmail.com) wrote:
>>> > > > > > > >
>>> > > > > > > > I've been doing an internal hwx deployment (HCP) recently
>>> for
>>> > > > testing
>>> > > > > > > > purposes. I have some ES tweaks to work out, but other than
>>> > that
>>> > > it
>>> > > > > > has
>>> > > > > > > > been running smoothly. Can you share any log info or
>>> > screenshots?
>>> > > > > > > >
>>> > > > > > > > Mike
>>> > > > > > > >
>>> > > > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <
>>> > ottobackwards@gmail.com
>>> > > >
>>> > > > > > wrote:
>>> > > > > > > >
>>> > > > > > > > > Hi,
>>> > > > > > > > > Should I see the repo tab in the ambari wizard for
>>> metron at
>>> > > this
>>> > > > > > > point?
>>> > > > > > > > > Asking for a friend who can’t get the wizard to work :)
>>> > > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > > --
>>> > >
>>> > > Jon
>>> > >
>>> > >
>>> >
>>>
>>>
>>
>
Re: Ambari Wizard: Repo Tab
Posted by Nick Allen <ni...@nickallen.org>.
I can create the JIRA. I capture the logs.
On Wed, Apr 26, 2017 at 4:48 PM, Nick Allen <ni...@nickallen.org> wrote:
> Yes, Otto. I just experienced that myself. It is a bug that we should
> create a JIRA for.
>
> I was able to work around it by just using "start" instead of "restart".
>
> On Wed, Apr 26, 2017 at 3:41 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
>> Ok, I did ifconfig, i mean ip a and got the right values
>> that and the brackets and it looks like es is up.
>>
>> Now my next problem is prob. a bug.
>>
>> Ambari thinks it needs to restart ‘metron’.
>> the enrichment restart crashes because we don’t catch the storm exception
>> that you are stopping a topology that was not running.
>>
>> The question is - is the state wrong _and_ the exception catch or not.
>>
>> I am guessing I should start a Jira for that one?
>>
>>
>>
>> On April 26, 2017 at 15:31:05, Otto Fowler (ottobackwards@gmail.com)
>> wrote:
>>
>> So now I’m getting errors for ETH0
>> I think that is one of the things that has to change on centos7?
>> Can you check what you have?
>>
>>
>>
>> On April 26, 2017 at 15:20:18, Nick Allen (nick@nickallen.org) wrote:
>>
>> Yes, I am also running on CentOS 7.
>>
>> On Wed, Apr 26, 2017 at 3:19 PM, Otto Fowler <ot...@gmail.com>
>> wrote:
>>
>>> Would i have to change that on Centos 7?
>>>
>>>
>>> On April 26, 2017 at 15:12:38, Nick Allen (nick@nickallen.org) wrote:
>>>
>>> Check on the `network_hosts` setting. I just ran into the same issue. I
>>> had to add brackets around it otherwise the file was not valid YAML and
>>> it
>>> would crash trying to load the config file.
>>>
>>> network.host: ["_lo:ipv4_","_eth0:ipv4_"]
>>>
>>> On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler <ot...@gmail.com>
>>> wrote:
>>>
>>> > I think it is literally crashing trying to load the config file
>>> >
>>> >
>>> >
>>> > On April 26, 2017 at 15:00:25, Michael Miklavcic (
>>> > michael.miklavcic@gmail.com) wrote:
>>> >
>>> > A couple more things you should check, given this setup:
>>> >
>>> > - 1 master node (not as data node)
>>> > - 2 data nodes
>>> >
>>> > I'd use something like this:
>>> >
>>> > - gateway_recover_after_data_nodes=1 or 2
>>> > - index_number_of_replicas=0 or 1 (2 data nodes, with 2 total copies of
>>> > the index - if you set this value to 2, it would need 1 more data node
>>> > to
>>> > attain 1 orig index + 2 replicas)
>>> >
>>> >
>>> >
>>> > On Wed, Apr 26, 2017 at 12:02 PM, Otto Fowler <ottobackwards@gmail.com
>>> >
>>> > wrote:
>>> >
>>> > > Actually never mind. I am not sure how to read the exception. Is it
>>> > > failing to read the configuration?
>>> > >
>>> > >
>>> > > On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwards@gmail.com)
>>> > > wrote:
>>> > >
>>> > > So the elasticsearch service is disabled. That is the issue. When ES
>>> > > calls get status ( service elasticsearch status ) it fails.
>>> > > I don’t know systemd too well, but does that mean we didn’t do the
>>> equiv
>>> > > of chkconfig on?
>>> > >
>>> > >
>>> > >
>>> > > On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com)
>>> > > wrote:
>>> > >
>>> > > Getting the same error in ES as before from what I can see
>>> > >
>>> > >
>>> > > On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com)
>>> > > wrote:
>>> > >
>>> > > I still can’t keep ES running, it dies.
>>> > > Also, I’m getting exceptions from enrichment trying to restart when
>>> the
>>> > > topology wasn’t running.
>>> > >
>>> > > Is there some coordination in this work? Should I just log jiras?
>>> > > If someone is already sussing out ES, we can sync up
>>> > >
>>> > >
>>> > >
>>> > > On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com)
>>> > wrote:
>>> > >
>>> > > I'm also interested to know why that's important at such a small
>>> scale.
>>> > >
>>> > > Jon
>>> > >
>>> > > On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com>
>>> > > wrote:
>>> > >
>>> > > > I am following
>>> > > >
>>> > > > https://community.hortonworks.com/articles/60805/deploying-
>>> > > a-fresh-metron-cluster-using-ambari-serv.html
>>> > > > I DO have master and data node together.
>>> > > > Why is that a problem?
>>> > > >
>>> > > > I will try again with master and data node separate.
>>> > > >
>>> > > >
>>> > > > On April 26, 2017 at 10:41:59, Michael Miklavcic (
>>> > > > michael.miklavcic@gmail.com) wrote:
>>> > > >
>>> > > > Hey Otto,
>>> > > >
>>> > > > How do you have the ES nodes configured? For a base install I would
>>> > > setup 1
>>> > > > master (NOT as data node) and 2 data nodes (NOT on the same node
>>> as the
>>> > > > master). This is the install configuration I got working. You can
>>> also
>>> > > > modify some configuration properties around master node as data
>>> node,
>>> > > index
>>> > > > replicas, and gateway recovery to get it working differently, but
>>> this
>>> > is
>>> > > > what will work OOTB with the default config settings from the
>>> mpack. If
>>> > > > you've already setup a master node and a data node on the same
>>> host,
>>> > > we'll
>>> > > > need to re-install.
>>> > > >
>>> > > > Mike
>>> > > >
>>> > > >
>>> > > > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <
>>> ottobackwards@gmail.com>
>>> > > > wrote:
>>> > > >
>>> > > > > File
>>> > > > > "/usr/lib/python2.6/site-packages/resource_management/
>>> > core/shell.py",
>>> > > > line
>>> > > > > 293, in _call
>>> > > > >
>>> > > > > raise ExecutionFailed(err_msg, code, out, err)
>>> > > > >
>>> > > > > ExecutionFailed: Execution of 'service elasticsearch status'
>>> returned
>>> > > 3.
>>> > > > ●
>>> > > > > elasticsearch.service - Elasticsearch
>>> > > > >
>>> > > > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
>>> > > > > disabled;
>>> > > > > vendor preset: disabled)
>>> > > > >
>>> > > > > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58
>>> EDT
>>> > > > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
>>> > > > >
>>> > > > > Docs: http://www.elastic.co
>>> > > > >
>>> > > > > Process: 16821 ExecStart=/usr/share/elasticse
>>> arch/bin/elasticsearch
>>> > > > > -Des.pidfile=${PID_DIR}/elasticsearch.pid
>>> > -Des.default.path.home=${ES_
>>> > > > > HOME}
>>> > > > > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_
>>> DIR}
>>> > > > > -Des.default.path.conf=${CONF_DIR} (code=exited,
>>> status=1/FAILURE)
>>> > > > >
>>> > > > > Process: 16819
>>> > > > > ExecStartPre=/usr/share/elasticsearch/bin/
>>> > > elasticsearch-systemd-pre-exec
>>> > > > > (code=exited, status=0/SUCCESS)
>>> > > > >
>>> > > > > Main PID: 16821 (code=exited, status=1/FAILURE)
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>> elasticsearch[16821]:
>>> > at
>>> > > > > org.elasticsearch.common.settings.Settings$Builder.
>>> > > > > loadFromStream(Settings.java:1080)
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>> elasticsearch[16821]:
>>> > at
>>> > > > > org.elasticsearch.common.settings.Settings$Builder.
>>> > > > > loadFromPath(Settings.java:1067)
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>> elasticsearch[16821]:
>>> > at
>>> > > > > org.elasticsearch.node.internal.InternalSettingsPreparer.
>>> > > > > prepareEnvironment(InternalSettingsPreparer.java:88)
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>> elasticsearch[16821]:
>>> > at
>>> > > > > org.elasticsearch.bootstrap.Bootstrap.initialSettings(
>>> > > Bootstrap.java:202)
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>> elasticsearch[16821]:
>>> > at
>>> > > > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>> elasticsearch[16821]:
>>> > at
>>> > > > > org.elasticsearch.bootstrap.Elasticsearch.main(
>>> > Elasticsearch.java:35)
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>>> elasticsearch[16821]:
>>> > > Refer
>>> > > > > to
>>> > > > > the log for complete error details.
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
>>> > > > > elasticsearch.service: main process exited, code=exited,
>>> > > status=1/FAILURE
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
>>> > > > > elasticsearch.service entered failed state.
>>> > > > >
>>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>>> 04-25%2022:14:58%20EDT
>>> > >
>>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
>>> > > > > elasticsearch.service failed.
>>> > > > >
>>> > > > > INFO 2017-04-26 08:55:19,962
>>> > > > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
>>> > > > >
>>> > > > >
>>> > > > > On April 26, 2017 at 08:27:27, Otto Fowler (
>>> ottobackwards@gmail.com)
>>> > > > > wrote:
>>> > > > >
>>> > > > > Can you describe what ES issues you are working on?
>>> > > > > Fixing the repos got everything installed, but my ES components
>>> don’t
>>> > > > stay
>>> > > > > running.
>>> > > > > I need to harvest the errors.
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > On April 25, 2017 at 16:46:00, Otto Fowler (
>>> ottobackwards@gmail.com)
>>> > > > > wrote:
>>> > > > >
>>> > > > > Nm. sorry. I fixed it.
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > On April 25, 2017 at 16:42:05, Otto Fowler (
>>> ottobackwards@gmail.com)
>>> > > > > wrote:
>>> > > > >
>>> > > > > Ok, now I see the repos in the ‘pick version’ screen, but it is
>>> > erring
>>> > > on
>>> > > > > the f://localrepo
>>> > > > > even though the folder exists, there is no repodata/repomd.xml.
>>> > > > >
>>> > > > > What is the command to create a local repo?
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > On April 25, 2017 at 16:05:17, Otto Fowler (
>>> ottobackwards@gmail.com)
>>> > > > > wrote:
>>> > > > >
>>> > > > > I was going by the HW community page.
>>> > > > >
>>> > > > > Ok, Let me try it
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com)
>>> > > wrote:
>>> > > > >
>>> > > > > That would do it. It requires 2.4.2+. I would have sworn I put
>>> that
>>> > in
>>> > > > the
>>> > > > > README, but I must have only annotated the PR. :(
>>> > > > >
>>> > > > > I'll get that in asap.
>>> > > > >
>>> > > > > -D...
>>> > > > >
>>> > > > >
>>> > > > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <
>>> > ottobackwards@gmail.com>
>>> > > > > wrote:
>>> > > > >
>>> > > > > > 2.4.1 as a matter of fact
>>> > > > > >
>>> > > > > >
>>> > > > > > On April 25, 2017 at 15:29:43, David Lyle (
>>> dlyle65535@gmail.com)
>>> > > > wrote:
>>> > > > > >
>>> > > > > > Any chance they're running Ambari < 2.4.2?
>>> > > > > >
>>> > > > > > -D...
>>> > > > > >
>>> > > > > >
>>> > > > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
>>> > > > > > michael.miklavcic@gmail.com> wrote:
>>> > > > > >
>>> > > > > > > Hey Otto, I don't have wizard screenshots in front of me
>>> right
>>> > now
>>> > > to
>>> > > > > > say
>>> > > > > > > for sure, but I do see a repoinfo.xml in the mpack. I
>>> haven't run
>>> > > > into
>>> > > > > > > anything like that, but next time I run through the install
>>> I can
>>> > > > take
>>> > > > > > > another look.
>>> > > > > > >
>>> > > > > > > https://github.com/apache/incubator-metron/blob/master/
>>> > > > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
>>> > > > > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.
>>> xml
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
>>> > > > ottobackwards@gmail.com
>>> > > > > >
>>> > > > > > > wrote:
>>> > > > > > >
>>> > > > > > > > stderr:
>>> > > > > > > >
>>> > > > > > > > Traceback (most recent call last):
>>> > > > > > > >
>>> > > > > > > > File "/var/lib/ambari-agent/cache/
>>> > common-services/ELASTICSEARCH/
>>> > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in
>>> <module>
>>> > > > > > > >
>>> > > > > > > > Elasticsearch().execute()
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > libraries/script/script.py",
>>> > > > > > > > line 280, in execute
>>> > > > > > > >
>>> > > > > > > > method(env)
>>> > > > > > > >
>>> > > > > > > > File "/var/lib/ambari-agent/cache/
>>> > common-services/ELASTICSEARCH/
>>> > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in
>>> install
>>> > > > > > > >
>>> > > > > > > > self.install_packages(env)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > libraries/script/script.py",
>>> > > > > > > > line 567, in install_packages
>>> > > > > > > >
>>> > > > > > > > retry_count=agent_stack_retry_count)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > core/base.py",
>>> > > > > > > > line 155, in __init__
>>> > > > > > > >
>>> > > > > > > > self.env.run()
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > core/environment.py",
>>> > > > > > > > line 160, in run
>>> > > > > > > >
>>> > > > > > > > self.run_action(resource, action)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > core/environment.py",
>>> > > > > > > > line 124, in run_action
>>> > > > > > > >
>>> > > > > > > > provider_action()
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > > core/providers/package/__init__.py", line 54, in
>>> > action_install
>>> > > > > > > >
>>> > > > > > > > self.install_package(package_name,
>>> self.resource.use_repos,
>>> > > > > > > > self.resource.skip_repos)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > > core/providers/package/yumrpm.py", line 49, in
>>> install_package
>>> > > > > > > >
>>> > > > > > > > self.checked_call_with_retries(cmd, sudo=True,
>>> > > > > > > > logoutput=self.get_logoutput())
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > > core/providers/package/__init__.py", line 83, in
>>> > > > > > > checked_call_with_retries
>>> > > > > > > >
>>> > > > > > > > return self._call_with_retries(cmd, is_checked=True,
>>> **kwargs)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > > core/providers/package/__init__.py", line 91, in
>>> > > _call_with_retries
>>> > > > > > > >
>>> > > > > > > > code, out = func(cmd, **kwargs)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > core/shell.py",
>>> > > > > > > > line 71, in inner
>>> > > > > > > >
>>> > > > > > > > result = function(command, **kwargs)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > core/shell.py",
>>> > > > > > > > line 93, in checked_call
>>> > > > > > > >
>>> > > > > > > > tries=tries, try_sleep=try_sleep)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > core/shell.py",
>>> > > > > > > > line 141, in _call_wrapper
>>> > > > > > > >
>>> > > > > > > > result = _call(command, **kwargs_copy)
>>> > > > > > > >
>>> > > > > > > > File "/usr/lib/python2.6/site-packa
>>> ges/resource_management/
>>> > > > > > > core/shell.py",
>>> > > > > > > > line 294, in _call
>>> > > > > > > >
>>> > > > > > > > raise Fail(err_msg)
>>> > > > > > > >
>>> > > > > > > > resource_management.core.exceptions.Fail: Execution of
>>> > > > '/usr/bin/yum
>>> > > > > > -d
>>> > > > > > > 0
>>> > > > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error:
>>> Nothing
>>> > > to
>>> > > > do
>>> > > > > > > >
>>> > > > > > > > stdout:
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,669
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Using
>>> > > > hadoop
>>> > > > > > > conf
>>> > > > > > > > dir: /usr/hdp/current/hadoop-client/conf
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,671
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > > Group['metron']
>>> > > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,673
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['metron']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,740
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > Group['livy']
>>> > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,741
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['livy']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,774
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > > > > Group['elasticsearch'] {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,775
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['elasticsearch']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,812
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > > Group['spark']
>>> > > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,812
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['spark']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,848
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > > > Group['zeppelin']
>>> > > > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,849
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['zeppelin']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,884
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > > Group['hadoop']
>>> > > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,885
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['hadoop']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,921
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > > Group['kibana']
>>> > > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,922
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['kibana']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,959
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > > > Group['users']
>>> > > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,960
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> > > > User['hive']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:48,961
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['hive']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,227
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > User['storm']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,227
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['storm']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,388
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > > > User['zookeeper']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,388
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['zookeeper']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,518
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > > > > User['infra-solr'] {'gid': 'hadoop',
>>> 'fetch_nonlocal_groups':
>>> > > True,
>>> > > > > > > > 'groups': [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,518
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['infra-solr']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,578
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > User['ams']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,578
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['ams']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,625
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > User['tez']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > [u'users']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,626
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['tez']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,660
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > > User['zeppelin']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,660
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['zeppelin']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,705
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > > User['metron']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,705
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['metron']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,760
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > User['livy']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,760
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['livy']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,803
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > > > > User['elasticsearch'] {'gid': 'hadoop',
>>> > 'fetch_nonlocal_groups':
>>> > > > > True,
>>> > > > > > > > 'groups': [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,804
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['elasticsearch']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,837
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > User['spark']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,837
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['spark']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,885
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > > > User['ambari-qa']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > [u'users']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,885
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['ambari-qa']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,928
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > User['flume']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,928
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['flume']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,972
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> > > > > User['kafka']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:49,973
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['kafka']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,025
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > User['hdfs']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,026
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['hdfs']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,075
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > User['yarn']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,075
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['yarn']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,125
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > User['kibana']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,126
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['kibana']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,175
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > User['mapred']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,176
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['mapred']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,223
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > User['hbase']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,224
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['hbase']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,271
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > User['hcat']
>>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>>> > > > > > [u'hadoop']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,272
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> Adding
>>> > > > user
>>> > > > > > > > User['hcat']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,320
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
>>> > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,333
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Writing
>>> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it
>>> > > doesn't
>>> > > > > > exist
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,334
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Changing
>>> > > > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh
>>> from 644
>>> > > to
>>> > > > > 555
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,335
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
>>> > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
>>> > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if':
>>> > > '(test
>>> > > > > > $(id
>>> > > > > > > -u
>>> > > > > > > > ambari-qa) -gt 1000) || (false)'}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,364
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Skipping
>>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
>>> > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
>>> > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to
>>> not_if
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,365
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
>>> > > 'create_parents':
>>> > > > > > True,
>>> > > > > > > > 'mode': 0775, 'cd_access': 'a'}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,366
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Creating
>>> > > > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't
>>> exist.
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,366
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Changing
>>> > > > > > owner
>>> > > > > > > > for /tmp/hbase-hbase from 0 to hbase
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,366
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Changing
>>> > > > > > > > permission for /tmp/hbase-hbase from 755 to 775
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,368
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
>>> > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,369
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
>>> > > > > > > >
>>> > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
>>> > > hbase-hbase']
>>> > > > > >
>>> > > > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,393
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Skipping
>>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
>>> > > > > > > >
>>> > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
>>> > > hbase-hbase']
>>> > > > > >
>>> > > > > > > > due to not_if
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,394
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > Group['hdfs']
>>> > > > > > {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,395
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> Adding
>>> > > > group
>>> > > > > > > > Group['hdfs']
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,430
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > User['hdfs']
>>> > > > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop',
>>> u'hdfs']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,431
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > Modifying
>>> > > > > > user
>>> > > > > > > > hdfs
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,474
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS
>>> > Type:
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,474
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > Directory['/etc/hadoop'] {'mode': 0755}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,474
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Creating
>>> > > > > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,475
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > Directory['/var/lib/ambari-age
>>> nt/tmp/hadoop_java_io_tmpdir']
>>> > > > > > {'owner':
>>> > > > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,476
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Creating
>>> > > > > > > > directory Directory['/var/lib/ambari-age
>>> nt/tmp/hadoop_java_io_
>>> > > > > tmpdir']
>>> > > > > >
>>> > > > > > > > since it doesn't exist.
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,476
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Changing
>>> > > > > > owner
>>> > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
>>> 0 to
>>> > > hdfs
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,476
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Changing
>>> > > > > > group
>>> > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
>>> 0 to
>>> > > > hadoop
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,477
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Changing
>>> > > > > > > > permission for /var/lib/ambari-agent/tmp/hado
>>> op_java_io_tmpdir
>>> > > from
>>> > > > > > 755
>>> > > > > > > > to 1777
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,477
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
>>> > > > > > > > {'create_parents': True}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,478
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > Creating
>>> > > > > > > > directory Directory['/var/lib/ambari-
>>> > > agent/tmp/AMBARI-artifacts/']
>>> > > > > > since
>>> > > > > > > > it doesn't exist.
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,479
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > > > > > File['/var/lib/ambari-agent/tm
>>> p/jdk-8u77-linux-x64.tar.gz']
>>> > > > > > {'content':
>>> > > > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
>>> > > > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
>>> > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:50,504
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>>> > > > Downloading
>>> > > > > > the
>>> > > > > > > > file from http://css-fx-met-poc-82:8080/
>>> resources//jdk-8u77-
>>> > > > > > > > linux-x64.tar.gz
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:53,793
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>>> > > > > > > > Directory['/usr/jdk64'] {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:53,793
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>>> > Creating
>>> > > > > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:53,794
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>>> > > > > > > Execute[('chmod',
>>> > > > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:12:53,824
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>>> > > > Execute['cd
>>> > > > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
>>> > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
>>> > > > > ambari-sudo.sh
>>> > > > > > cp
>>> > > > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/*
>>> /usr/jdk64'] {}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,439
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
>>> > {'action':
>>> > > > > > > > ['delete']}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,440
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > Removing
>>> > > > > > > > directory Directory['/var/lib/ambari-
>>> > agent/tmp/jdk_tmp_1BRMsS']
>>> > > and
>>> > > > > > all
>>> > > > > > > > its content
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,620
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
>>> > > 'cd_access':
>>> > > > > > 'a'}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,621
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > > Execute[('chmod',
>>> > > > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,682
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > Initializing
>>> > > > > > 2
>>> > > > > > > > repositories
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,683
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
>>> > > > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0',
>>> 'action':
>>> > > > > > ['create'],
>>> > > > > > > > 'components': [u'HDP', 'main'], 'repo_template':
>>> > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
>>> > > > > > > > %}mirrorlist={{mirror_list}}{% else
>>> %}baseurl={{base_url}}{%
>>> > > endif
>>> > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
>>> 'HDP',
>>> > > > > > > > 'mirror_list': None}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,709
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content':
>>> > > InlineTemplate(...)}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,711
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > Writing
>>> > > > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,711
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
>>> > > > http://public-repo-1.
>>> > > >
>>> > > > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7',
>>> 'action':
>>> > > > > > ['create'],
>>> > > > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
>>> > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
>>> > > > > > > > %}mirrorlist={{mirror_list}}{% else
>>> %}baseurl={{base_url}}{%
>>> > > endif
>>> > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
>>> > > > 'HDP-UTILS',
>>> > > > > > > > 'mirror_list': None}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,717
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
>>> > > > > > InlineTemplate(...)}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,717
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > Writing
>>> > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't
>>> > exist
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,718
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > > > > Package['unzip']
>>> > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:00,846
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>>> > > Installing
>>> > > > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:03,413
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>>> > > > > > Package['curl']
>>> > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:03,429
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>>> > Skipping
>>> > > > > > > > installation of existing package curl
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:03,430
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>>> > > > > > > > Package['hdp-select'] {'retry_on_repo_unavailability':
>>> False,
>>> > > > > > > > 'retry_count': 5}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:03,451
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>>> > > Installing
>>> > > > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
>>> > > hdp-select')
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:04,948
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
>>> > Install
>>> > > ES
>>> > > > > > Data
>>> > > > > > > > Node
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:04,952
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
>>> > > > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_
>>> > unavailability':
>>> > > > > > False,
>>> > > > > > > > 'retry_count': 5}
>>> > > > > > > >
>>> > > > > > > > 2017-04-25 14:13:05,086
>>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> -
>>> > > Installing
>>> > > > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y
>>> install
>>> > > > > > > > elasticsearch-2.3.3')
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > Command failed after 1 tries
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
>>> > > > ottobackwards@gmail.com)
>>> > > >
>>> > > > > > > > wrote:
>>> > > > > > > >
>>> > > > > > > > Yes, I’m going to run through again and get the logs. The
>>> basic
>>> > > > issue
>>> > > > > > > is,
>>> > > > > > > > no repo tab -> if I should have one,
>>> > > > > > > > and the ES yum command failing with nothing to do, which
>>> stops
>>> > > > > > deployment
>>> > > > > > > > on all the nodes.
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
>>> > > > > > > > michael.miklavcic@gmail.com) wrote:
>>> > > > > > > >
>>> > > > > > > > I've been doing an internal hwx deployment (HCP) recently
>>> for
>>> > > > testing
>>> > > > > > > > purposes. I have some ES tweaks to work out, but other than
>>> > that
>>> > > it
>>> > > > > > has
>>> > > > > > > > been running smoothly. Can you share any log info or
>>> > screenshots?
>>> > > > > > > >
>>> > > > > > > > Mike
>>> > > > > > > >
>>> > > > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <
>>> > ottobackwards@gmail.com
>>> > > >
>>> > > > > > wrote:
>>> > > > > > > >
>>> > > > > > > > > Hi,
>>> > > > > > > > > Should I see the repo tab in the ambari wizard for
>>> metron at
>>> > > this
>>> > > > > > > point?
>>> > > > > > > > > Asking for a friend who can’t get the wizard to work :)
>>> > > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > > --
>>> > >
>>> > > Jon
>>> > >
>>> > >
>>> >
>>>
>>>
>>
>
Re: Ambari Wizard: Repo Tab
Posted by Nick Allen <ni...@nickallen.org>.
Yes, Otto. I just experienced that myself. It is a bug that we should
create a JIRA for.
I was able to work around it by just using "start" instead of "restart".
On Wed, Apr 26, 2017 at 3:41 PM, Otto Fowler <ot...@gmail.com>
wrote:
> Ok, I did ifconfig, i mean ip a and got the right values
> that and the brackets and it looks like es is up.
>
> Now my next problem is prob. a bug.
>
> Ambari thinks it needs to restart ‘metron’.
> the enrichment restart crashes because we don’t catch the storm exception
> that you are stopping a topology that was not running.
>
> The question is - is the state wrong _and_ the exception catch or not.
>
> I am guessing I should start a Jira for that one?
>
>
>
> On April 26, 2017 at 15:31:05, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> So now I’m getting errors for ETH0
> I think that is one of the things that has to change on centos7?
> Can you check what you have?
>
>
>
> On April 26, 2017 at 15:20:18, Nick Allen (nick@nickallen.org) wrote:
>
> Yes, I am also running on CentOS 7.
>
> On Wed, Apr 26, 2017 at 3:19 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
>> Would i have to change that on Centos 7?
>>
>>
>> On April 26, 2017 at 15:12:38, Nick Allen (nick@nickallen.org) wrote:
>>
>> Check on the `network_hosts` setting. I just ran into the same issue. I
>> had to add brackets around it otherwise the file was not valid YAML and it
>> would crash trying to load the config file.
>>
>> network.host: ["_lo:ipv4_","_eth0:ipv4_"]
>>
>> On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler <ot...@gmail.com>
>> wrote:
>>
>> > I think it is literally crashing trying to load the config file
>> >
>> >
>> >
>> > On April 26, 2017 at 15:00:25, Michael Miklavcic (
>> > michael.miklavcic@gmail.com) wrote:
>> >
>> > A couple more things you should check, given this setup:
>> >
>> > - 1 master node (not as data node)
>> > - 2 data nodes
>> >
>> > I'd use something like this:
>> >
>> > - gateway_recover_after_data_nodes=1 or 2
>> > - index_number_of_replicas=0 or 1 (2 data nodes, with 2 total copies of
>> > the index - if you set this value to 2, it would need 1 more data node
>> > to
>> > attain 1 orig index + 2 replicas)
>> >
>> >
>> >
>> > On Wed, Apr 26, 2017 at 12:02 PM, Otto Fowler <ot...@gmail.com>
>> > wrote:
>> >
>> > > Actually never mind. I am not sure how to read the exception. Is it
>> > > failing to read the configuration?
>> > >
>> > >
>> > > On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwards@gmail.com)
>> > > wrote:
>> > >
>> > > So the elasticsearch service is disabled. That is the issue. When ES
>> > > calls get status ( service elasticsearch status ) it fails.
>> > > I don’t know systemd too well, but does that mean we didn’t do the
>> equiv
>> > > of chkconfig on?
>> > >
>> > >
>> > >
>> > > On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com)
>> > > wrote:
>> > >
>> > > Getting the same error in ES as before from what I can see
>> > >
>> > >
>> > > On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com)
>> > > wrote:
>> > >
>> > > I still can’t keep ES running, it dies.
>> > > Also, I’m getting exceptions from enrichment trying to restart when
>> the
>> > > topology wasn’t running.
>> > >
>> > > Is there some coordination in this work? Should I just log jiras?
>> > > If someone is already sussing out ES, we can sync up
>> > >
>> > >
>> > >
>> > > On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com)
>> > wrote:
>> > >
>> > > I'm also interested to know why that's important at such a small
>> scale.
>> > >
>> > > Jon
>> > >
>> > > On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com>
>> > > wrote:
>> > >
>> > > > I am following
>> > > >
>> > > > https://community.hortonworks.com/articles/60805/deploying-
>> > > a-fresh-metron-cluster-using-ambari-serv.html
>> > > > I DO have master and data node together.
>> > > > Why is that a problem?
>> > > >
>> > > > I will try again with master and data node separate.
>> > > >
>> > > >
>> > > > On April 26, 2017 at 10:41:59, Michael Miklavcic (
>> > > > michael.miklavcic@gmail.com) wrote:
>> > > >
>> > > > Hey Otto,
>> > > >
>> > > > How do you have the ES nodes configured? For a base install I would
>> > > setup 1
>> > > > master (NOT as data node) and 2 data nodes (NOT on the same node as
>> the
>> > > > master). This is the install configuration I got working. You can
>> also
>> > > > modify some configuration properties around master node as data
>> node,
>> > > index
>> > > > replicas, and gateway recovery to get it working differently, but
>> this
>> > is
>> > > > what will work OOTB with the default config settings from the
>> mpack. If
>> > > > you've already setup a master node and a data node on the same host,
>> > > we'll
>> > > > need to re-install.
>> > > >
>> > > > Mike
>> > > >
>> > > >
>> > > > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <
>> ottobackwards@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > File
>> > > > > "/usr/lib/python2.6/site-packages/resource_management/
>> > core/shell.py",
>> > > > line
>> > > > > 293, in _call
>> > > > >
>> > > > > raise ExecutionFailed(err_msg, code, out, err)
>> > > > >
>> > > > > ExecutionFailed: Execution of 'service elasticsearch status'
>> returned
>> > > 3.
>> > > > ●
>> > > > > elasticsearch.service - Elasticsearch
>> > > > >
>> > > > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
>> > > > > disabled;
>> > > > > vendor preset: disabled)
>> > > > >
>> > > > > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58
>> EDT
>> > > > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
>> > > > >
>> > > > > Docs: http://www.elastic.co
>> > > > >
>> > > > > Process: 16821 ExecStart=/usr/share/elasticse
>> arch/bin/elasticsearch
>> > > > > -Des.pidfile=${PID_DIR}/elasticsearch.pid
>> > -Des.default.path.home=${ES_
>> > > > > HOME}
>> > > > > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_
>> DIR}
>> > > > > -Des.default.path.conf=${CONF_DIR} (code=exited,
>> status=1/FAILURE)
>> > > > >
>> > > > > Process: 16819
>> > > > > ExecStartPre=/usr/share/elasticsearch/bin/
>> > > elasticsearch-systemd-pre-exec
>> > > > > (code=exited, status=0/SUCCESS)
>> > > > >
>> > > > > Main PID: 16821 (code=exited, status=1/FAILURE)
>> > > > >
>> > > > >
>> > > > >
>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>> 04-25%2022:14:58%20EDT
>> > >
>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>> elasticsearch[16821]:
>> > at
>> > > > > org.elasticsearch.common.settings.Settings$Builder.
>> > > > > loadFromStream(Settings.java:1080)
>> > > > >
>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>> 04-25%2022:14:58%20EDT
>> > >
>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>> elasticsearch[16821]:
>> > at
>> > > > > org.elasticsearch.common.settings.Settings$Builder.
>> > > > > loadFromPath(Settings.java:1067)
>> > > > >
>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>> 04-25%2022:14:58%20EDT
>> > >
>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>> elasticsearch[16821]:
>> > at
>> > > > > org.elasticsearch.node.internal.InternalSettingsPreparer.
>> > > > > prepareEnvironment(InternalSettingsPreparer.java:88)
>> > > > >
>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>> 04-25%2022:14:58%20EDT
>> > >
>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>> elasticsearch[16821]:
>> > at
>> > > > > org.elasticsearch.bootstrap.Bootstrap.initialSettings(
>> > > Bootstrap.java:202)
>> > > > >
>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>> 04-25%2022:14:58%20EDT
>> > >
>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>> elasticsearch[16821]:
>> > at
>> > > > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
>> > > > >
>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>> 04-25%2022:14:58%20EDT
>> > >
>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>> elasticsearch[16821]:
>> > at
>> > > > > org.elasticsearch.bootstrap.Elasticsearch.main(
>> > Elasticsearch.java:35)
>> > > > >
>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>> 04-25%2022:14:58%20EDT
>> > >
>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
>> elasticsearch[16821]:
>> > > Refer
>> > > > > to
>> > > > > the log for complete error details.
>> > > > >
>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>> 04-25%2022:14:58%20EDT
>> > >
>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
>> > > > > elasticsearch.service: main process exited, code=exited,
>> > > status=1/FAILURE
>> > > > >
>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>> 04-25%2022:14:58%20EDT
>> > >
>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
>> > > > > elasticsearch.service entered failed state.
>> > > > >
>> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
>> 04-25%2022:14:58%20EDT
>> > >
>> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
>> > > > > elasticsearch.service failed.
>> > > > >
>> > > > > INFO 2017-04-26 08:55:19,962
>> > > > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
>> > > > >
>> > > > >
>> > > > > On April 26, 2017 at 08:27:27, Otto Fowler (
>> ottobackwards@gmail.com)
>> > > > > wrote:
>> > > > >
>> > > > > Can you describe what ES issues you are working on?
>> > > > > Fixing the repos got everything installed, but my ES components
>> don’t
>> > > > stay
>> > > > > running.
>> > > > > I need to harvest the errors.
>> > > > >
>> > > > >
>> > > > >
>> > > > > On April 25, 2017 at 16:46:00, Otto Fowler (
>> ottobackwards@gmail.com)
>> > > > > wrote:
>> > > > >
>> > > > > Nm. sorry. I fixed it.
>> > > > >
>> > > > >
>> > > > >
>> > > > > On April 25, 2017 at 16:42:05, Otto Fowler (
>> ottobackwards@gmail.com)
>> > > > > wrote:
>> > > > >
>> > > > > Ok, now I see the repos in the ‘pick version’ screen, but it is
>> > erring
>> > > on
>> > > > > the f://localrepo
>> > > > > even though the folder exists, there is no repodata/repomd.xml.
>> > > > >
>> > > > > What is the command to create a local repo?
>> > > > >
>> > > > >
>> > > > >
>> > > > > On April 25, 2017 at 16:05:17, Otto Fowler (
>> ottobackwards@gmail.com)
>> > > > > wrote:
>> > > > >
>> > > > > I was going by the HW community page.
>> > > > >
>> > > > > Ok, Let me try it
>> > > > >
>> > > > >
>> > > > >
>> > > > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com)
>> > > wrote:
>> > > > >
>> > > > > That would do it. It requires 2.4.2+. I would have sworn I put
>> that
>> > in
>> > > > the
>> > > > > README, but I must have only annotated the PR. :(
>> > > > >
>> > > > > I'll get that in asap.
>> > > > >
>> > > > > -D...
>> > > > >
>> > > > >
>> > > > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <
>> > ottobackwards@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > 2.4.1 as a matter of fact
>> > > > > >
>> > > > > >
>> > > > > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com
>> )
>> > > > wrote:
>> > > > > >
>> > > > > > Any chance they're running Ambari < 2.4.2?
>> > > > > >
>> > > > > > -D...
>> > > > > >
>> > > > > >
>> > > > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
>> > > > > > michael.miklavcic@gmail.com> wrote:
>> > > > > >
>> > > > > > > Hey Otto, I don't have wizard screenshots in front of me right
>> > now
>> > > to
>> > > > > > say
>> > > > > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't
>> run
>> > > > into
>> > > > > > > anything like that, but next time I run through the install I
>> can
>> > > > take
>> > > > > > > another look.
>> > > > > > >
>> > > > > > > https://github.com/apache/incubator-metron/blob/master/
>> > > > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
>> > > > > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.
>> xml
>> > > > > > >
>> > > > > > >
>> > > > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
>> > > > ottobackwards@gmail.com
>> > > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > stderr:
>> > > > > > > >
>> > > > > > > > Traceback (most recent call last):
>> > > > > > > >
>> > > > > > > > File "/var/lib/ambari-agent/cache/
>> > common-services/ELASTICSEARCH/
>> > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in
>> <module>
>> > > > > > > >
>> > > > > > > > Elasticsearch().execute()
>> > > > > > > >
>> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > > > > > libraries/script/script.py",
>> > > > > > > > line 280, in execute
>> > > > > > > >
>> > > > > > > > method(env)
>> > > > > > > >
>> > > > > > > > File "/var/lib/ambari-agent/cache/
>> > common-services/ELASTICSEARCH/
>> > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in
>> install
>> > > > > > > >
>> > > > > > > > self.install_packages(env)
>> > > > > > > >
>> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > > > > > libraries/script/script.py",
>> > > > > > > > line 567, in install_packages
>> > > > > > > >
>> > > > > > > > retry_count=agent_stack_retry_count)
>> > > > > > > >
>> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > > > > > core/base.py",
>> > > > > > > > line 155, in __init__
>> > > > > > > >
>> > > > > > > > self.env.run()
>> > > > > > > >
>> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > > > > > core/environment.py",
>> > > > > > > > line 160, in run
>> > > > > > > >
>> > > > > > > > self.run_action(resource, action)
>> > > > > > > >
>> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > > > > > core/environment.py",
>> > > > > > > > line 124, in run_action
>> > > > > > > >
>> > > > > > > > provider_action()
>> > > > > > > >
>> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > > > > > > core/providers/package/__init__.py", line 54, in
>> > action_install
>> > > > > > > >
>> > > > > > > > self.install_package(package_name, self.resource.use_repos,
>> > > > > > > > self.resource.skip_repos)
>> > > > > > > >
>> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > > > > > > core/providers/package/yumrpm.py", line 49, in
>> install_package
>> > > > > > > >
>> > > > > > > > self.checked_call_with_retries(cmd, sudo=True,
>> > > > > > > > logoutput=self.get_logoutput())
>> > > > > > > >
>> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > > > > > > core/providers/package/__init__.py", line 83, in
>> > > > > > > checked_call_with_retries
>> > > > > > > >
>> > > > > > > > return self._call_with_retries(cmd, is_checked=True,
>> **kwargs)
>> > > > > > > >
>> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > > > > > > core/providers/package/__init__.py", line 91, in
>> > > _call_with_retries
>> > > > > > > >
>> > > > > > > > code, out = func(cmd, **kwargs)
>> > > > > > > >
>> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > > > > > core/shell.py",
>> > > > > > > > line 71, in inner
>> > > > > > > >
>> > > > > > > > result = function(command, **kwargs)
>> > > > > > > >
>> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > > > > > core/shell.py",
>> > > > > > > > line 93, in checked_call
>> > > > > > > >
>> > > > > > > > tries=tries, try_sleep=try_sleep)
>> > > > > > > >
>> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > > > > > core/shell.py",
>> > > > > > > > line 141, in _call_wrapper
>> > > > > > > >
>> > > > > > > > result = _call(command, **kwargs_copy)
>> > > > > > > >
>> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > > > > > core/shell.py",
>> > > > > > > > line 294, in _call
>> > > > > > > >
>> > > > > > > > raise Fail(err_msg)
>> > > > > > > >
>> > > > > > > > resource_management.core.exceptions.Fail: Execution of
>> > > > '/usr/bin/yum
>> > > > > > -d
>> > > > > > > 0
>> > > > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error:
>> Nothing
>> > > to
>> > > > do
>> > > > > > > >
>> > > > > > > > stdout:
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,669
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Using
>> > > > hadoop
>> > > > > > > conf
>> > > > > > > > dir: /usr/hdp/current/hadoop-client/conf
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,671
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> > > > > > Group['metron']
>> > > > > > > {}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,673
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Adding
>> > > > group
>> > > > > > > > Group['metron']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,740
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> > > > > Group['livy']
>> > > > > > {}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,741
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Adding
>> > > > group
>> > > > > > > > Group['livy']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,774
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> > > > > > > > Group['elasticsearch'] {}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,775
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Adding
>> > > > group
>> > > > > > > > Group['elasticsearch']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,812
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> > > > > > Group['spark']
>> > > > > > > {}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,812
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Adding
>> > > > group
>> > > > > > > > Group['spark']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,848
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> > > > > > > Group['zeppelin']
>> > > > > > > > {}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,849
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Adding
>> > > > group
>> > > > > > > > Group['zeppelin']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,884
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> > > > > > Group['hadoop']
>> > > > > > > {}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,885
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Adding
>> > > > group
>> > > > > > > > Group['hadoop']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,921
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> > > > > > Group['kibana']
>> > > > > > > {}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,922
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Adding
>> > > > group
>> > > > > > > > Group['kibana']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,959
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> > > > > > Group['users']
>> > > > > > > {}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,960
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> > > > User['hive']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:48,961
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['hive']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,227
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > > > User['storm']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,227
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['storm']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,388
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > > > > > User['zookeeper']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,388
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['zookeeper']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,518
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > > > > > > User['infra-solr'] {'gid': 'hadoop',
>> 'fetch_nonlocal_groups':
>> > > True,
>> > > > > > > > 'groups': [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,518
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['infra-solr']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,578
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > > User['ams']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,578
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['ams']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,625
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > > User['tez']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > [u'users']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,626
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['tez']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,660
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > > > > User['zeppelin']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,660
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['zeppelin']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,705
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > > > > User['metron']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,705
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['metron']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,760
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > > User['livy']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,760
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['livy']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,803
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > > > > > > User['elasticsearch'] {'gid': 'hadoop',
>> > 'fetch_nonlocal_groups':
>> > > > > True,
>> > > > > > > > 'groups': [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,804
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['elasticsearch']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,837
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > > > User['spark']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,837
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['spark']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,885
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > > > > > User['ambari-qa']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > [u'users']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,885
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['ambari-qa']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,928
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > > > User['flume']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,928
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['flume']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,972
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > > > User['kafka']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:49,973
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['kafka']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,025
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > User['hdfs']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,026
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['hdfs']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,075
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > User['yarn']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,075
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['yarn']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,125
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > > > User['kibana']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,126
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['kibana']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,175
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > > > User['mapred']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,176
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['mapred']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,223
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > > User['hbase']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,224
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['hbase']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,271
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > User['hcat']
>> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> > > > > > [u'hadoop']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,272
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> Adding
>> > > > user
>> > > > > > > > User['hcat']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,320
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
>> > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,333
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > Writing
>> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it
>> > > doesn't
>> > > > > > exist
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,334
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > Changing
>> > > > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from
>> 644
>> > > to
>> > > > > 555
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,335
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
>> > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
>> > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if':
>> > > '(test
>> > > > > > $(id
>> > > > > > > -u
>> > > > > > > > ambari-qa) -gt 1000) || (false)'}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,364
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > Skipping
>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
>> > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
>> > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to
>> not_if
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,365
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
>> > > 'create_parents':
>> > > > > > True,
>> > > > > > > > 'mode': 0775, 'cd_access': 'a'}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,366
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > Creating
>> > > > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't
>> exist.
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,366
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > Changing
>> > > > > > owner
>> > > > > > > > for /tmp/hbase-hbase from 0 to hbase
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,366
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > Changing
>> > > > > > > > permission for /tmp/hbase-hbase from 755 to 775
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,368
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
>> > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,369
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
>> > > > > > > >
>> > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
>> > > hbase-hbase']
>> > > > > >
>> > > > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,393
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > Skipping
>> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
>> > > > > > > >
>> > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
>> > > hbase-hbase']
>> > > > > >
>> > > > > > > > due to not_if
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,394
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > > Group['hdfs']
>> > > > > > {}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,395
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> Adding
>> > > > group
>> > > > > > > > Group['hdfs']
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,430
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > User['hdfs']
>> > > > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop',
>> u'hdfs']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,431
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > Modifying
>> > > > > > user
>> > > > > > > > hdfs
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,474
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS
>> > Type:
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,474
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > > > > > Directory['/etc/hadoop'] {'mode': 0755}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,474
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > Creating
>> > > > > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,475
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > > > > > Directory['/var/lib/ambari-age
>> nt/tmp/hadoop_java_io_tmpdir']
>> > > > > > {'owner':
>> > > > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,476
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > Creating
>> > > > > > > > directory Directory['/var/lib/ambari-age
>> nt/tmp/hadoop_java_io_
>> > > > > tmpdir']
>> > > > > >
>> > > > > > > > since it doesn't exist.
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,476
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > Changing
>> > > > > > owner
>> > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0
>> to
>> > > hdfs
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,476
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > Changing
>> > > > > > group
>> > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0
>> to
>> > > > hadoop
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,477
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > Changing
>> > > > > > > > permission for /var/lib/ambari-agent/tmp/hado
>> op_java_io_tmpdir
>> > > from
>> > > > > > 755
>> > > > > > > > to 1777
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,477
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
>> > > > > > > > {'create_parents': True}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,478
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > Creating
>> > > > > > > > directory Directory['/var/lib/ambari-
>> > > agent/tmp/AMBARI-artifacts/']
>> > > > > > since
>> > > > > > > > it doesn't exist.
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,479
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
>> > > > > > {'content':
>> > > > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
>> > > > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
>> > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:50,504
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > > Downloading
>> > > > > > the
>> > > > > > > > file from http://css-fx-met-poc-82:8080/
>> resources//jdk-8u77-
>> > > > > > > > linux-x64.tar.gz
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:53,793
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>> > > > > > > > Directory['/usr/jdk64'] {}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:53,793
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>> > Creating
>> > > > > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:53,794
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>> > > > > > > Execute[('chmod',
>> > > > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:12:53,824
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>> > > > Execute['cd
>> > > > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
>> > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
>> > > > > ambari-sudo.sh
>> > > > > > cp
>> > > > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/*
>> /usr/jdk64'] {}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:00,439
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
>> > {'action':
>> > > > > > > > ['delete']}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:00,440
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > Removing
>> > > > > > > > directory Directory['/var/lib/ambari-
>> > agent/tmp/jdk_tmp_1BRMsS']
>> > > and
>> > > > > > all
>> > > > > > > > its content
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:00,620
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
>> > > 'cd_access':
>> > > > > > 'a'}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:00,621
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > > > > > Execute[('chmod',
>> > > > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:00,682
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > > Initializing
>> > > > > > 2
>> > > > > > > > repositories
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:00,683
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
>> > > > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
>> > > > > > ['create'],
>> > > > > > > > 'components': [u'HDP', 'main'], 'repo_template':
>> > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
>> > > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
>> > > endif
>> > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
>> 'HDP',
>> > > > > > > > 'mirror_list': None}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:00,709
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content':
>> > > InlineTemplate(...)}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:00,711
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > Writing
>> > > > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:00,711
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
>> > > > http://public-repo-1.
>> > > >
>> > > > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7',
>> 'action':
>> > > > > > ['create'],
>> > > > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
>> > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
>> > > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
>> > > endif
>> > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
>> > > > 'HDP-UTILS',
>> > > > > > > > 'mirror_list': None}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:00,717
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
>> > > > > > InlineTemplate(...)}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:00,717
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > Writing
>> > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't
>> > exist
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:00,718
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > > > > Package['unzip']
>> > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:00,846
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > Installing
>> > > > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:03,413
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>> > > > > > Package['curl']
>> > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:03,429
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>> > Skipping
>> > > > > > > > installation of existing package curl
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:03,430
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>> > > > > > > > Package['hdp-select'] {'retry_on_repo_unavailability':
>> False,
>> > > > > > > > 'retry_count': 5}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:03,451
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>> > > Installing
>> > > > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
>> > > hdp-select')
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:04,948
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
>> > Install
>> > > ES
>> > > > > > Data
>> > > > > > > > Node
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:04,952
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
>> > > > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_
>> > unavailability':
>> > > > > > False,
>> > > > > > > > 'retry_count': 5}
>> > > > > > > >
>> > > > > > > > 2017-04-25 14:13:05,086
>> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> -
>> > > Installing
>> > > > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y
>> install
>> > > > > > > > elasticsearch-2.3.3')
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Command failed after 1 tries
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
>> > > > ottobackwards@gmail.com)
>> > > >
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > Yes, I’m going to run through again and get the logs. The
>> basic
>> > > > issue
>> > > > > > > is,
>> > > > > > > > no repo tab -> if I should have one,
>> > > > > > > > and the ES yum command failing with nothing to do, which
>> stops
>> > > > > > deployment
>> > > > > > > > on all the nodes.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
>> > > > > > > > michael.miklavcic@gmail.com) wrote:
>> > > > > > > >
>> > > > > > > > I've been doing an internal hwx deployment (HCP) recently
>> for
>> > > > testing
>> > > > > > > > purposes. I have some ES tweaks to work out, but other than
>> > that
>> > > it
>> > > > > > has
>> > > > > > > > been running smoothly. Can you share any log info or
>> > screenshots?
>> > > > > > > >
>> > > > > > > > Mike
>> > > > > > > >
>> > > > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <
>> > ottobackwards@gmail.com
>> > > >
>> > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Hi,
>> > > > > > > > > Should I see the repo tab in the ambari wizard for metron
>> at
>> > > this
>> > > > > > > point?
>> > > > > > > > > Asking for a friend who can’t get the wizard to work :)
>> > > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > > --
>> > >
>> > > Jon
>> > >
>> > >
>> >
>>
>>
>
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
Ok, I did ifconfig, i mean ip a and got the right values
that and the brackets and it looks like es is up.
Now my next problem is prob. a bug.
Ambari thinks it needs to restart ‘metron’.
the enrichment restart crashes because we don’t catch the storm exception
that you are stopping a topology that was not running.
The question is - is the state wrong _and_ the exception catch or not.
I am guessing I should start a Jira for that one?
On April 26, 2017 at 15:31:05, Otto Fowler (ottobackwards@gmail.com) wrote:
So now I’m getting errors for ETH0
I think that is one of the things that has to change on centos7?
Can you check what you have?
On April 26, 2017 at 15:20:18, Nick Allen (nick@nickallen.org) wrote:
Yes, I am also running on CentOS 7.
On Wed, Apr 26, 2017 at 3:19 PM, Otto Fowler <ot...@gmail.com>
wrote:
> Would i have to change that on Centos 7?
>
>
> On April 26, 2017 at 15:12:38, Nick Allen (nick@nickallen.org) wrote:
>
> Check on the `network_hosts` setting. I just ran into the same issue. I
> had to add brackets around it otherwise the file was not valid YAML and it
> would crash trying to load the config file.
>
> network.host: ["_lo:ipv4_","_eth0:ipv4_"]
>
> On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > I think it is literally crashing trying to load the config file
> >
> >
> >
> > On April 26, 2017 at 15:00:25, Michael Miklavcic (
> > michael.miklavcic@gmail.com) wrote:
> >
> > A couple more things you should check, given this setup:
> >
> > - 1 master node (not as data node)
> > - 2 data nodes
> >
> > I'd use something like this:
> >
> > - gateway_recover_after_data_nodes=1 or 2
> > - index_number_of_replicas=0 or 1 (2 data nodes, with 2 total copies of
> > the index - if you set this value to 2, it would need 1 more data node
> > to
> > attain 1 orig index + 2 replicas)
> >
> >
> >
> > On Wed, Apr 26, 2017 at 12:02 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > Actually never mind. I am not sure how to read the exception. Is it
> > > failing to read the configuration?
> > >
> > >
> > > On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > So the elasticsearch service is disabled. That is the issue. When ES
> > > calls get status ( service elasticsearch status ) it fails.
> > > I don’t know systemd too well, but does that mean we didn’t do the
> equiv
> > > of chkconfig on?
> > >
> > >
> > >
> > > On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > Getting the same error in ES as before from what I can see
> > >
> > >
> > > On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > I still can’t keep ES running, it dies.
> > > Also, I’m getting exceptions from enrichment trying to restart when the
> > > topology wasn’t running.
> > >
> > > Is there some coordination in this work? Should I just log jiras?
> > > If someone is already sussing out ES, we can sync up
> > >
> > >
> > >
> > > On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com)
> > wrote:
> > >
> > > I'm also interested to know why that's important at such a small scale.
> > >
> > > Jon
> > >
> > > On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com>
> > > wrote:
> > >
> > > > I am following
> > > >
> > > > https://community.hortonworks.com/articles/60805/deploying-
> > > a-fresh-metron-cluster-using-ambari-serv.html
> > > > I DO have master and data node together.
> > > > Why is that a problem?
> > > >
> > > > I will try again with master and data node separate.
> > > >
> > > >
> > > > On April 26, 2017 at 10:41:59, Michael Miklavcic (
> > > > michael.miklavcic@gmail.com) wrote:
> > > >
> > > > Hey Otto,
> > > >
> > > > How do you have the ES nodes configured? For a base install I would
> > > setup 1
> > > > master (NOT as data node) and 2 data nodes (NOT on the same node as
> the
> > > > master). This is the install configuration I got working. You can
> also
> > > > modify some configuration properties around master node as data node,
> > > index
> > > > replicas, and gateway recovery to get it working differently, but
> this
> > is
> > > > what will work OOTB with the default config settings from the mpack.
> If
> > > > you've already setup a master node and a data node on the same host,
> > > we'll
> > > > need to re-install.
> > > >
> > > > Mike
> > > >
> > > >
> > > > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <
> ottobackwards@gmail.com>
> > > > wrote:
> > > >
> > > > > File
> > > > > "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > > line
> > > > > 293, in _call
> > > > >
> > > > > raise ExecutionFailed(err_msg, code, out, err)
> > > > >
> > > > > ExecutionFailed: Execution of 'service elasticsearch status'
> returned
> > > 3.
> > > > ●
> > > > > elasticsearch.service - Elasticsearch
> > > > >
> > > > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> > > > > disabled;
> > > > > vendor preset: disabled)
> > > > >
> > > > > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58
> EDT
> > > > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
> > > > >
> > > > > Docs: http://www.elastic.co
> > > > >
> > > > > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/
> elasticsearch
> > > > > -Des.pidfile=${PID_DIR}/elasticsearch.pid
> > -Des.default.path.home=${ES_
> > > > > HOME}
> > > > > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_
> DIR}
> > > > > -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
> > > > >
> > > > > Process: 16819
> > > > > ExecStartPre=/usr/share/elasticsearch/bin/
> > > elasticsearch-systemd-pre-exec
> > > > > (code=exited, status=0/SUCCESS)
> > > > >
> > > > > Main PID: 16821 (code=exited, status=1/FAILURE)
> > > > >
> > > > >
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > > loadFromStream(Settings.java:1080)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > > loadFromPath(Settings.java:1067)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.node.internal.InternalSettingsPreparer.
> > > > > prepareEnvironment(InternalSettingsPreparer.java:88)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.bootstrap.Bootstrap.initialSettings(
> > > Bootstrap.java:202)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.bootstrap.Elasticsearch.main(
> > Elasticsearch.java:35)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > > Refer
> > > > > to
> > > > > the log for complete error details.
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > > elasticsearch.service: main process exited, code=exited,
> > > status=1/FAILURE
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> > > > > elasticsearch.service entered failed state.
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > > elasticsearch.service failed.
> > > > >
> > > > > INFO 2017-04-26 08:55:19,962
> > > > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
> > > > >
> > > > >
> > > > > On April 26, 2017 at 08:27:27, Otto Fowler (
> ottobackwards@gmail.com)
> > > > > wrote:
> > > > >
> > > > > Can you describe what ES issues you are working on?
> > > > > Fixing the repos got everything installed, but my ES components
> don’t
> > > > stay
> > > > > running.
> > > > > I need to harvest the errors.
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 16:46:00, Otto Fowler (
> ottobackwards@gmail.com)
> > > > > wrote:
> > > > >
> > > > > Nm. sorry. I fixed it.
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 16:42:05, Otto Fowler (
> ottobackwards@gmail.com)
> > > > > wrote:
> > > > >
> > > > > Ok, now I see the repos in the ‘pick version’ screen, but it is
> > erring
> > > on
> > > > > the f://localrepo
> > > > > even though the folder exists, there is no repodata/repomd.xml.
> > > > >
> > > > > What is the command to create a local repo?
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 16:05:17, Otto Fowler (
> ottobackwards@gmail.com)
> > > > > wrote:
> > > > >
> > > > > I was going by the HW community page.
> > > > >
> > > > > Ok, Let me try it
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com)
> > > wrote:
> > > > >
> > > > > That would do it. It requires 2.4.2+. I would have sworn I put that
> > in
> > > > the
> > > > > README, but I must have only annotated the PR. :(
> > > > >
> > > > > I'll get that in asap.
> > > > >
> > > > > -D...
> > > > >
> > > > >
> > > > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <
> > ottobackwards@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > 2.4.1 as a matter of fact
> > > > > >
> > > > > >
> > > > > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com)
> > > > wrote:
> > > > > >
> > > > > > Any chance they're running Ambari < 2.4.2?
> > > > > >
> > > > > > -D...
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > > > > > michael.miklavcic@gmail.com> wrote:
> > > > > >
> > > > > > > Hey Otto, I don't have wizard screenshots in front of me right
> > now
> > > to
> > > > > > say
> > > > > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't
> run
> > > > into
> > > > > > > anything like that, but next time I run through the install I
> can
> > > > take
> > > > > > > another look.
> > > > > > >
> > > > > > > https://github.com/apache/incubator-metron/blob/master/
> > > > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > > > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/
> repoinfo.xml
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
> > > > ottobackwards@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > stderr:
> > > > > > > >
> > > > > > > > Traceback (most recent call last):
> > > > > > > >
> > > > > > > > File "/var/lib/ambari-agent/cache/
> > common-services/ELASTICSEARCH/
> > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in
> <module>
> > > > > > > >
> > > > > > > > Elasticsearch().execute()
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > libraries/script/script.py",
> > > > > > > > line 280, in execute
> > > > > > > >
> > > > > > > > method(env)
> > > > > > > >
> > > > > > > > File "/var/lib/ambari-agent/cache/
> > common-services/ELASTICSEARCH/
> > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > > > > > > >
> > > > > > > > self.install_packages(env)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > libraries/script/script.py",
> > > > > > > > line 567, in install_packages
> > > > > > > >
> > > > > > > > retry_count=agent_stack_retry_count)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/base.py",
> > > > > > > > line 155, in __init__
> > > > > > > >
> > > > > > > > self.env.run()
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/environment.py",
> > > > > > > > line 160, in run
> > > > > > > >
> > > > > > > > self.run_action(resource, action)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/environment.py",
> > > > > > > > line 124, in run_action
> > > > > > > >
> > > > > > > > provider_action()
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > > core/providers/package/__init__.py", line 54, in
> > action_install
> > > > > > > >
> > > > > > > > self.install_package(package_name, self.resource.use_repos,
> > > > > > > > self.resource.skip_repos)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > > core/providers/package/yumrpm.py", line 49, in
> install_package
> > > > > > > >
> > > > > > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > > > > > logoutput=self.get_logoutput())
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > > core/providers/package/__init__.py", line 83, in
> > > > > > > checked_call_with_retries
> > > > > > > >
> > > > > > > > return self._call_with_retries(cmd, is_checked=True,
> **kwargs)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > > core/providers/package/__init__.py", line 91, in
> > > _call_with_retries
> > > > > > > >
> > > > > > > > code, out = func(cmd, **kwargs)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/shell.py",
> > > > > > > > line 71, in inner
> > > > > > > >
> > > > > > > > result = function(command, **kwargs)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/shell.py",
> > > > > > > > line 93, in checked_call
> > > > > > > >
> > > > > > > > tries=tries, try_sleep=try_sleep)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/shell.py",
> > > > > > > > line 141, in _call_wrapper
> > > > > > > >
> > > > > > > > result = _call(command, **kwargs_copy)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/shell.py",
> > > > > > > > line 294, in _call
> > > > > > > >
> > > > > > > > raise Fail(err_msg)
> > > > > > > >
> > > > > > > > resource_management.core.exceptions.Fail: Execution of
> > > > '/usr/bin/yum
> > > > > > -d
> > > > > > > 0
> > > > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error:
> Nothing
> > > to
> > > > do
> > > > > > > >
> > > > > > > > stdout:
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,669
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Using
> > > > hadoop
> > > > > > > conf
> > > > > > > > dir: /usr/hdp/current/hadoop-client/conf
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,671
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['metron']
> > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,673
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['metron']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,740
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['livy']
> > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,741
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['livy']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,774
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > > Group['elasticsearch'] {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,775
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['elasticsearch']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,812
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['spark']
> > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,812
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['spark']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,848
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['zeppelin']
> > > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,849
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['zeppelin']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,884
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['hadoop']
> > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,885
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['hadoop']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,921
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['kibana']
> > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,922
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['kibana']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,959
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['users']
> > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,960
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > User['hive']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,961
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > user
> > > > > > > > User['hive']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,227
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['storm']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,227
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['storm']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,388
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['zookeeper']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,388
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['zookeeper']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,518
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> > > True,
> > > > > > > > 'groups': [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,518
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['infra-solr']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,578
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['ams']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,578
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['ams']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,625
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['tez']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'users']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,626
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['tez']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,660
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['zeppelin']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,660
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['zeppelin']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,705
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['metron']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,705
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['metron']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,760
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['livy']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,760
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['livy']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,803
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > > User['elasticsearch'] {'gid': 'hadoop',
> > 'fetch_nonlocal_groups':
> > > > > True,
> > > > > > > > 'groups': [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,804
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['elasticsearch']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,837
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['spark']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,837
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['spark']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,885
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['ambari-qa']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'users']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,885
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['ambari-qa']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,928
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['flume']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,928
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['flume']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,972
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['kafka']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,973
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['kafka']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,025
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['hdfs']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,026
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['hdfs']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,075
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['yarn']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,075
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['yarn']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,125
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > User['kibana']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,126
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['kibana']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,175
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > User['mapred']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,176
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['mapred']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,223
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['hbase']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,224
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['hbase']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,271
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['hcat']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,272
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['hcat']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,320
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,333
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Writing
> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it
> > > doesn't
> > > > > > exist
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,334
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from
> 644
> > > to
> > > > > 555
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,335
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if':
> > > '(test
> > > > > > $(id
> > > > > > > -u
> > > > > > > > ambari-qa) -gt 1000) || (false)'}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,364
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Skipping
> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to
> not_if
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,365
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
> > > 'create_parents':
> > > > > > True,
> > > > > > > > 'mode': 0775, 'cd_access': 'a'}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,366
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Creating
> > > > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't
> exist.
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,366
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > owner
> > > > > > > > for /tmp/hbase-hbase from 0 to hbase
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,366
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > > > permission for /tmp/hbase-hbase from 755 to 775
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,368
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,369
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > > >
> > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > > hbase-hbase']
> > > > > >
> > > > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,393
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Skipping
> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > > >
> > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > > hbase-hbase']
> > > > > >
> > > > > > > > due to not_if
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,394
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Group['hdfs']
> > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,395
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['hdfs']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,430
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['hdfs']
> > > > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop',
> u'hdfs']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,431
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Modifying
> > > > > > user
> > > > > > > > hdfs
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,474
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS
> > Type:
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,474
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,474
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Creating
> > > > > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,475
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > > > > > {'owner':
> > > > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,476
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Creating
> > > > > > > > directory Directory['/var/lib/ambari-
> agent/tmp/hadoop_java_io_
> > > > > tmpdir']
> > > > > >
> > > > > > > > since it doesn't exist.
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,476
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > owner
> > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0
> to
> > > hdfs
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,476
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > group
> > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0
> to
> > > > hadoop
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,477
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > > > permission for /var/lib/ambari-agent/tmp/
> hadoop_java_io_tmpdir
> > > from
> > > > > > 755
> > > > > > > > to 1777
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,477
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > > > > > {'create_parents': True}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,478
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Creating
> > > > > > > > directory Directory['/var/lib/ambari-
> > > agent/tmp/AMBARI-artifacts/']
> > > > > > since
> > > > > > > > it doesn't exist.
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,479
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > > > > > {'content':
> > > > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,504
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Downloading
> > > > > > the
> > > > > > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > > > > > linux-x64.tar.gz
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:53,793
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > > > Directory['/usr/jdk64'] {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:53,793
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > Creating
> > > > > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:53,794
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > > Execute[('chmod',
> > > > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:53,824
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > Execute['cd
> > > > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> > > > > ambari-sudo.sh
> > > > > > cp
> > > > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64']
> {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,439
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
> > {'action':
> > > > > > > > ['delete']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,440
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Removing
> > > > > > > > directory Directory['/var/lib/ambari-
> > agent/tmp/jdk_tmp_1BRMsS']
> > > and
> > > > > > all
> > > > > > > > its content
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,620
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
> > > 'cd_access':
> > > > > > 'a'}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,621
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > Execute[('chmod',
> > > > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,682
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Initializing
> > > > > > 2
> > > > > > > > repositories
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,683
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > > > > > ['create'],
> > > > > > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> > > endif
> > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> 'HDP',
> > > > > > > > 'mirror_list': None}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,709
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content':
> > > InlineTemplate(...)}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,711
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Writing
> > > > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,711
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
> > > > http://public-repo-1.
> > > >
> > > > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> > > > > > ['create'],
> > > > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> > > endif
> > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> > > > 'HDP-UTILS',
> > > > > > > > 'mirror_list': None}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,717
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > > > > > InlineTemplate(...)}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,717
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Writing
> > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't
> > exist
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,718
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > Package['unzip']
> > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,846
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Installing
> > > > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:03,413
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > > Package['curl']
> > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:03,429
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > Skipping
> > > > > > > > installation of existing package curl
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:03,430
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > > > > Package['hdp-select'] {'retry_on_repo_unavailability':
> False,
> > > > > > > > 'retry_count': 5}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:03,451
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Installing
> > > > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
> > > hdp-select')
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:04,948
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > Install
> > > ES
> > > > > > Data
> > > > > > > > Node
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:04,952
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_
> > unavailability':
> > > > > > False,
> > > > > > > > 'retry_count': 5}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:05,086
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> -
> > > Installing
> > > > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y
> install
> > > > > > > > elasticsearch-2.3.3')
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Command failed after 1 tries
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
> > > > ottobackwards@gmail.com)
> > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > Yes, I’m going to run through again and get the logs. The
> basic
> > > > issue
> > > > > > > is,
> > > > > > > > no repo tab -> if I should have one,
> > > > > > > > and the ES yum command failing with nothing to do, which
> stops
> > > > > > deployment
> > > > > > > > on all the nodes.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > > > > > michael.miklavcic@gmail.com) wrote:
> > > > > > > >
> > > > > > > > I've been doing an internal hwx deployment (HCP) recently for
> > > > testing
> > > > > > > > purposes. I have some ES tweaks to work out, but other than
> > that
> > > it
> > > > > > has
> > > > > > > > been running smoothly. Can you share any log info or
> > screenshots?
> > > > > > > >
> > > > > > > > Mike
> > > > > > > >
> > > > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <
> > ottobackwards@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > > Should I see the repo tab in the ambari wizard for metron
> at
> > > this
> > > > > > > point?
> > > > > > > > > Asking for a friend who can’t get the wizard to work :)
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > > --
> > >
> > > Jon
> > >
> > >
> >
>
>
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
So now I’m getting errors for ETH0
I think that is one of the things that has to change on centos7?
Can you check what you have?
On April 26, 2017 at 15:20:18, Nick Allen (nick@nickallen.org) wrote:
Yes, I am also running on CentOS 7.
On Wed, Apr 26, 2017 at 3:19 PM, Otto Fowler <ot...@gmail.com>
wrote:
> Would i have to change that on Centos 7?
>
>
> On April 26, 2017 at 15:12:38, Nick Allen (nick@nickallen.org) wrote:
>
> Check on the `network_hosts` setting. I just ran into the same issue. I
> had to add brackets around it otherwise the file was not valid YAML and it
> would crash trying to load the config file.
>
> network.host: ["_lo:ipv4_","_eth0:ipv4_"]
>
> On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > I think it is literally crashing trying to load the config file
> >
> >
> >
> > On April 26, 2017 at 15:00:25, Michael Miklavcic (
> > michael.miklavcic@gmail.com) wrote:
> >
> > A couple more things you should check, given this setup:
> >
> > - 1 master node (not as data node)
> > - 2 data nodes
> >
> > I'd use something like this:
> >
> > - gateway_recover_after_data_nodes=1 or 2
> > - index_number_of_replicas=0 or 1 (2 data nodes, with 2 total copies of
> > the index - if you set this value to 2, it would need 1 more data node
> > to
> > attain 1 orig index + 2 replicas)
> >
> >
> >
> > On Wed, Apr 26, 2017 at 12:02 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > Actually never mind. I am not sure how to read the exception. Is it
> > > failing to read the configuration?
> > >
> > >
> > > On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > So the elasticsearch service is disabled. That is the issue. When ES
> > > calls get status ( service elasticsearch status ) it fails.
> > > I don’t know systemd too well, but does that mean we didn’t do the
> equiv
> > > of chkconfig on?
> > >
> > >
> > >
> > > On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > Getting the same error in ES as before from what I can see
> > >
> > >
> > > On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > I still can’t keep ES running, it dies.
> > > Also, I’m getting exceptions from enrichment trying to restart when the
> > > topology wasn’t running.
> > >
> > > Is there some coordination in this work? Should I just log jiras?
> > > If someone is already sussing out ES, we can sync up
> > >
> > >
> > >
> > > On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com)
> > wrote:
> > >
> > > I'm also interested to know why that's important at such a small scale.
> > >
> > > Jon
> > >
> > > On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com>
> > > wrote:
> > >
> > > > I am following
> > > >
> > > > https://community.hortonworks.com/articles/60805/deploying-
> > > a-fresh-metron-cluster-using-ambari-serv.html
> > > > I DO have master and data node together.
> > > > Why is that a problem?
> > > >
> > > > I will try again with master and data node separate.
> > > >
> > > >
> > > > On April 26, 2017 at 10:41:59, Michael Miklavcic (
> > > > michael.miklavcic@gmail.com) wrote:
> > > >
> > > > Hey Otto,
> > > >
> > > > How do you have the ES nodes configured? For a base install I would
> > > setup 1
> > > > master (NOT as data node) and 2 data nodes (NOT on the same node as
> the
> > > > master). This is the install configuration I got working. You can
> also
> > > > modify some configuration properties around master node as data node,
> > > index
> > > > replicas, and gateway recovery to get it working differently, but
> this
> > is
> > > > what will work OOTB with the default config settings from the mpack.
> If
> > > > you've already setup a master node and a data node on the same host,
> > > we'll
> > > > need to re-install.
> > > >
> > > > Mike
> > > >
> > > >
> > > > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <
> ottobackwards@gmail.com>
> > > > wrote:
> > > >
> > > > > File
> > > > > "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > > line
> > > > > 293, in _call
> > > > >
> > > > > raise ExecutionFailed(err_msg, code, out, err)
> > > > >
> > > > > ExecutionFailed: Execution of 'service elasticsearch status'
> returned
> > > 3.
> > > > ●
> > > > > elasticsearch.service - Elasticsearch
> > > > >
> > > > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> > > > > disabled;
> > > > > vendor preset: disabled)
> > > > >
> > > > > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58
> EDT
> > > > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
> > > > >
> > > > > Docs: http://www.elastic.co
> > > > >
> > > > > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/
> elasticsearch
> > > > > -Des.pidfile=${PID_DIR}/elasticsearch.pid
> > -Des.default.path.home=${ES_
> > > > > HOME}
> > > > > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_
> DIR}
> > > > > -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
> > > > >
> > > > > Process: 16819
> > > > > ExecStartPre=/usr/share/elasticsearch/bin/
> > > elasticsearch-systemd-pre-exec
> > > > > (code=exited, status=0/SUCCESS)
> > > > >
> > > > > Main PID: 16821 (code=exited, status=1/FAILURE)
> > > > >
> > > > >
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > > loadFromStream(Settings.java:1080)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > > loadFromPath(Settings.java:1067)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.node.internal.InternalSettingsPreparer.
> > > > > prepareEnvironment(InternalSettingsPreparer.java:88)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.bootstrap.Bootstrap.initialSettings(
> > > Bootstrap.java:202)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.bootstrap.Elasticsearch.main(
> > Elasticsearch.java:35)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > > Refer
> > > > > to
> > > > > the log for complete error details.
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > > elasticsearch.service: main process exited, code=exited,
> > > status=1/FAILURE
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> > > > > elasticsearch.service entered failed state.
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > > elasticsearch.service failed.
> > > > >
> > > > > INFO 2017-04-26 08:55:19,962
> > > > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
> > > > >
> > > > >
> > > > > On April 26, 2017 at 08:27:27, Otto Fowler (
> ottobackwards@gmail.com)
> > > > > wrote:
> > > > >
> > > > > Can you describe what ES issues you are working on?
> > > > > Fixing the repos got everything installed, but my ES components
> don’t
> > > > stay
> > > > > running.
> > > > > I need to harvest the errors.
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 16:46:00, Otto Fowler (
> ottobackwards@gmail.com)
> > > > > wrote:
> > > > >
> > > > > Nm. sorry. I fixed it.
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 16:42:05, Otto Fowler (
> ottobackwards@gmail.com)
> > > > > wrote:
> > > > >
> > > > > Ok, now I see the repos in the ‘pick version’ screen, but it is
> > erring
> > > on
> > > > > the f://localrepo
> > > > > even though the folder exists, there is no repodata/repomd.xml.
> > > > >
> > > > > What is the command to create a local repo?
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 16:05:17, Otto Fowler (
> ottobackwards@gmail.com)
> > > > > wrote:
> > > > >
> > > > > I was going by the HW community page.
> > > > >
> > > > > Ok, Let me try it
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com)
> > > wrote:
> > > > >
> > > > > That would do it. It requires 2.4.2+. I would have sworn I put that
> > in
> > > > the
> > > > > README, but I must have only annotated the PR. :(
> > > > >
> > > > > I'll get that in asap.
> > > > >
> > > > > -D...
> > > > >
> > > > >
> > > > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <
> > ottobackwards@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > 2.4.1 as a matter of fact
> > > > > >
> > > > > >
> > > > > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com)
> > > > wrote:
> > > > > >
> > > > > > Any chance they're running Ambari < 2.4.2?
> > > > > >
> > > > > > -D...
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > > > > > michael.miklavcic@gmail.com> wrote:
> > > > > >
> > > > > > > Hey Otto, I don't have wizard screenshots in front of me right
> > now
> > > to
> > > > > > say
> > > > > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't
> run
> > > > into
> > > > > > > anything like that, but next time I run through the install I
> can
> > > > take
> > > > > > > another look.
> > > > > > >
> > > > > > > https://github.com/apache/incubator-metron/blob/master/
> > > > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > > > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/
> repoinfo.xml
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
> > > > ottobackwards@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > stderr:
> > > > > > > >
> > > > > > > > Traceback (most recent call last):
> > > > > > > >
> > > > > > > > File "/var/lib/ambari-agent/cache/
> > common-services/ELASTICSEARCH/
> > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in
> <module>
> > > > > > > >
> > > > > > > > Elasticsearch().execute()
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > libraries/script/script.py",
> > > > > > > > line 280, in execute
> > > > > > > >
> > > > > > > > method(env)
> > > > > > > >
> > > > > > > > File "/var/lib/ambari-agent/cache/
> > common-services/ELASTICSEARCH/
> > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > > > > > > >
> > > > > > > > self.install_packages(env)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > libraries/script/script.py",
> > > > > > > > line 567, in install_packages
> > > > > > > >
> > > > > > > > retry_count=agent_stack_retry_count)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/base.py",
> > > > > > > > line 155, in __init__
> > > > > > > >
> > > > > > > > self.env.run()
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/environment.py",
> > > > > > > > line 160, in run
> > > > > > > >
> > > > > > > > self.run_action(resource, action)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/environment.py",
> > > > > > > > line 124, in run_action
> > > > > > > >
> > > > > > > > provider_action()
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > > core/providers/package/__init__.py", line 54, in
> > action_install
> > > > > > > >
> > > > > > > > self.install_package(package_name, self.resource.use_repos,
> > > > > > > > self.resource.skip_repos)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > > core/providers/package/yumrpm.py", line 49, in
> install_package
> > > > > > > >
> > > > > > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > > > > > logoutput=self.get_logoutput())
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > > core/providers/package/__init__.py", line 83, in
> > > > > > > checked_call_with_retries
> > > > > > > >
> > > > > > > > return self._call_with_retries(cmd, is_checked=True,
> **kwargs)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > > core/providers/package/__init__.py", line 91, in
> > > _call_with_retries
> > > > > > > >
> > > > > > > > code, out = func(cmd, **kwargs)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/shell.py",
> > > > > > > > line 71, in inner
> > > > > > > >
> > > > > > > > result = function(command, **kwargs)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/shell.py",
> > > > > > > > line 93, in checked_call
> > > > > > > >
> > > > > > > > tries=tries, try_sleep=try_sleep)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/shell.py",
> > > > > > > > line 141, in _call_wrapper
> > > > > > > >
> > > > > > > > result = _call(command, **kwargs_copy)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/shell.py",
> > > > > > > > line 294, in _call
> > > > > > > >
> > > > > > > > raise Fail(err_msg)
> > > > > > > >
> > > > > > > > resource_management.core.exceptions.Fail: Execution of
> > > > '/usr/bin/yum
> > > > > > -d
> > > > > > > 0
> > > > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error:
> Nothing
> > > to
> > > > do
> > > > > > > >
> > > > > > > > stdout:
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,669
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Using
> > > > hadoop
> > > > > > > conf
> > > > > > > > dir: /usr/hdp/current/hadoop-client/conf
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,671
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['metron']
> > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,673
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['metron']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,740
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['livy']
> > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,741
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['livy']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,774
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > > Group['elasticsearch'] {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,775
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['elasticsearch']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,812
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['spark']
> > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,812
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['spark']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,848
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['zeppelin']
> > > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,849
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['zeppelin']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,884
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['hadoop']
> > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,885
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['hadoop']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,921
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['kibana']
> > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,922
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['kibana']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,959
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['users']
> > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,960
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > User['hive']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,961
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > user
> > > > > > > > User['hive']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,227
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['storm']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,227
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['storm']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,388
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['zookeeper']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,388
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['zookeeper']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,518
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> > > True,
> > > > > > > > 'groups': [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,518
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['infra-solr']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,578
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['ams']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,578
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['ams']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,625
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['tez']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'users']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,626
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['tez']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,660
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['zeppelin']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,660
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['zeppelin']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,705
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['metron']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,705
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['metron']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,760
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['livy']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,760
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['livy']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,803
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > > User['elasticsearch'] {'gid': 'hadoop',
> > 'fetch_nonlocal_groups':
> > > > > True,
> > > > > > > > 'groups': [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,804
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['elasticsearch']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,837
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['spark']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,837
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['spark']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,885
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['ambari-qa']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'users']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,885
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['ambari-qa']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,928
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['flume']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,928
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['flume']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,972
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['kafka']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,973
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['kafka']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,025
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['hdfs']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,026
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['hdfs']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,075
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['yarn']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,075
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['yarn']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,125
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > User['kibana']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,126
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['kibana']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,175
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > User['mapred']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,176
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['mapred']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,223
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['hbase']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,224
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['hbase']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,271
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['hcat']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,272
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['hcat']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,320
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,333
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Writing
> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it
> > > doesn't
> > > > > > exist
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,334
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from
> 644
> > > to
> > > > > 555
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,335
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if':
> > > '(test
> > > > > > $(id
> > > > > > > -u
> > > > > > > > ambari-qa) -gt 1000) || (false)'}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,364
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Skipping
> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to
> not_if
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,365
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
> > > 'create_parents':
> > > > > > True,
> > > > > > > > 'mode': 0775, 'cd_access': 'a'}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,366
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Creating
> > > > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't
> exist.
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,366
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > owner
> > > > > > > > for /tmp/hbase-hbase from 0 to hbase
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,366
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > > > permission for /tmp/hbase-hbase from 755 to 775
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,368
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,369
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > > >
> > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > > hbase-hbase']
> > > > > >
> > > > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,393
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Skipping
> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > > >
> > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > > hbase-hbase']
> > > > > >
> > > > > > > > due to not_if
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,394
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Group['hdfs']
> > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,395
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['hdfs']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,430
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['hdfs']
> > > > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop',
> u'hdfs']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,431
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Modifying
> > > > > > user
> > > > > > > > hdfs
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,474
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS
> > Type:
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,474
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,474
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Creating
> > > > > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,475
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > > > > > {'owner':
> > > > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,476
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Creating
> > > > > > > > directory Directory['/var/lib/ambari-
> agent/tmp/hadoop_java_io_
> > > > > tmpdir']
> > > > > >
> > > > > > > > since it doesn't exist.
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,476
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > owner
> > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0
> to
> > > hdfs
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,476
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > group
> > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0
> to
> > > > hadoop
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,477
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > > > permission for /var/lib/ambari-agent/tmp/
> hadoop_java_io_tmpdir
> > > from
> > > > > > 755
> > > > > > > > to 1777
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,477
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > > > > > {'create_parents': True}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,478
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Creating
> > > > > > > > directory Directory['/var/lib/ambari-
> > > agent/tmp/AMBARI-artifacts/']
> > > > > > since
> > > > > > > > it doesn't exist.
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,479
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > > > > > {'content':
> > > > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,504
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Downloading
> > > > > > the
> > > > > > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > > > > > linux-x64.tar.gz
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:53,793
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > > > Directory['/usr/jdk64'] {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:53,793
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > Creating
> > > > > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:53,794
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > > Execute[('chmod',
> > > > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:53,824
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > Execute['cd
> > > > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> > > > > ambari-sudo.sh
> > > > > > cp
> > > > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64']
> {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,439
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
> > {'action':
> > > > > > > > ['delete']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,440
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Removing
> > > > > > > > directory Directory['/var/lib/ambari-
> > agent/tmp/jdk_tmp_1BRMsS']
> > > and
> > > > > > all
> > > > > > > > its content
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,620
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
> > > 'cd_access':
> > > > > > 'a'}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,621
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > Execute[('chmod',
> > > > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,682
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Initializing
> > > > > > 2
> > > > > > > > repositories
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,683
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > > > > > ['create'],
> > > > > > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> > > endif
> > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> 'HDP',
> > > > > > > > 'mirror_list': None}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,709
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content':
> > > InlineTemplate(...)}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,711
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Writing
> > > > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,711
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
> > > > http://public-repo-1.
> > > >
> > > > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> > > > > > ['create'],
> > > > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> > > endif
> > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> > > > 'HDP-UTILS',
> > > > > > > > 'mirror_list': None}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,717
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > > > > > InlineTemplate(...)}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,717
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Writing
> > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't
> > exist
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,718
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > Package['unzip']
> > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,846
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Installing
> > > > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:03,413
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > > Package['curl']
> > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:03,429
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > Skipping
> > > > > > > > installation of existing package curl
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:03,430
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > > > > Package['hdp-select'] {'retry_on_repo_unavailability':
> False,
> > > > > > > > 'retry_count': 5}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:03,451
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Installing
> > > > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
> > > hdp-select')
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:04,948
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > Install
> > > ES
> > > > > > Data
> > > > > > > > Node
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:04,952
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_
> > unavailability':
> > > > > > False,
> > > > > > > > 'retry_count': 5}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:05,086
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> -
> > > Installing
> > > > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y
> install
> > > > > > > > elasticsearch-2.3.3')
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Command failed after 1 tries
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
> > > > ottobackwards@gmail.com)
> > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > Yes, I’m going to run through again and get the logs. The
> basic
> > > > issue
> > > > > > > is,
> > > > > > > > no repo tab -> if I should have one,
> > > > > > > > and the ES yum command failing with nothing to do, which
> stops
> > > > > > deployment
> > > > > > > > on all the nodes.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > > > > > michael.miklavcic@gmail.com) wrote:
> > > > > > > >
> > > > > > > > I've been doing an internal hwx deployment (HCP) recently for
> > > > testing
> > > > > > > > purposes. I have some ES tweaks to work out, but other than
> > that
> > > it
> > > > > > has
> > > > > > > > been running smoothly. Can you share any log info or
> > screenshots?
> > > > > > > >
> > > > > > > > Mike
> > > > > > > >
> > > > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <
> > ottobackwards@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > > Should I see the repo tab in the ambari wizard for metron
> at
> > > this
> > > > > > > point?
> > > > > > > > > Asking for a friend who can’t get the wizard to work :)
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > > --
> > >
> > > Jon
> > >
> > >
> >
>
>
Re: Ambari Wizard: Repo Tab
Posted by Nick Allen <ni...@nickallen.org>.
Yes, I am also running on CentOS 7.
On Wed, Apr 26, 2017 at 3:19 PM, Otto Fowler <ot...@gmail.com>
wrote:
> Would i have to change that on Centos 7?
>
>
> On April 26, 2017 at 15:12:38, Nick Allen (nick@nickallen.org) wrote:
>
> Check on the `network_hosts` setting. I just ran into the same issue. I
> had to add brackets around it otherwise the file was not valid YAML and it
> would crash trying to load the config file.
>
> network.host: ["_lo:ipv4_","_eth0:ipv4_"]
>
> On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > I think it is literally crashing trying to load the config file
> >
> >
> >
> > On April 26, 2017 at 15:00:25, Michael Miklavcic (
> > michael.miklavcic@gmail.com) wrote:
> >
> > A couple more things you should check, given this setup:
> >
> > - 1 master node (not as data node)
> > - 2 data nodes
> >
> > I'd use something like this:
> >
> > - gateway_recover_after_data_nodes=1 or 2
> > - index_number_of_replicas=0 or 1 (2 data nodes, with 2 total copies of
> > the index - if you set this value to 2, it would need 1 more data node
> > to
> > attain 1 orig index + 2 replicas)
> >
> >
> >
> > On Wed, Apr 26, 2017 at 12:02 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > Actually never mind. I am not sure how to read the exception. Is it
> > > failing to read the configuration?
> > >
> > >
> > > On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > So the elasticsearch service is disabled. That is the issue. When ES
> > > calls get status ( service elasticsearch status ) it fails.
> > > I don’t know systemd too well, but does that mean we didn’t do the
> equiv
> > > of chkconfig on?
> > >
> > >
> > >
> > > On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > Getting the same error in ES as before from what I can see
> > >
> > >
> > > On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > I still can’t keep ES running, it dies.
> > > Also, I’m getting exceptions from enrichment trying to restart when
> the
> > > topology wasn’t running.
> > >
> > > Is there some coordination in this work? Should I just log jiras?
> > > If someone is already sussing out ES, we can sync up
> > >
> > >
> > >
> > > On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com)
> > wrote:
> > >
> > > I'm also interested to know why that's important at such a small
> scale.
> > >
> > > Jon
> > >
> > > On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com>
> > > wrote:
> > >
> > > > I am following
> > > >
> > > > https://community.hortonworks.com/articles/60805/deploying-
> > > a-fresh-metron-cluster-using-ambari-serv.html
> > > > I DO have master and data node together.
> > > > Why is that a problem?
> > > >
> > > > I will try again with master and data node separate.
> > > >
> > > >
> > > > On April 26, 2017 at 10:41:59, Michael Miklavcic (
> > > > michael.miklavcic@gmail.com) wrote:
> > > >
> > > > Hey Otto,
> > > >
> > > > How do you have the ES nodes configured? For a base install I would
> > > setup 1
> > > > master (NOT as data node) and 2 data nodes (NOT on the same node as
> the
> > > > master). This is the install configuration I got working. You can
> also
> > > > modify some configuration properties around master node as data
> node,
> > > index
> > > > replicas, and gateway recovery to get it working differently, but
> this
> > is
> > > > what will work OOTB with the default config settings from the mpack.
> If
> > > > you've already setup a master node and a data node on the same host,
> > > we'll
> > > > need to re-install.
> > > >
> > > > Mike
> > > >
> > > >
> > > > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <
> ottobackwards@gmail.com>
> > > > wrote:
> > > >
> > > > > File
> > > > > "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > > line
> > > > > 293, in _call
> > > > >
> > > > > raise ExecutionFailed(err_msg, code, out, err)
> > > > >
> > > > > ExecutionFailed: Execution of 'service elasticsearch status'
> returned
> > > 3.
> > > > ●
> > > > > elasticsearch.service - Elasticsearch
> > > > >
> > > > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> > > > > disabled;
> > > > > vendor preset: disabled)
> > > > >
> > > > > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58
> EDT
> > > > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
> > > > >
> > > > > Docs: http://www.elastic.co
> > > > >
> > > > > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch
>
> > > > > -Des.pidfile=${PID_DIR}/elasticsearch.pid
> > -Des.default.path.home=${ES_
> > > > > HOME}
> > > > > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR}
>
> > > > > -Des.default.path.conf=${CONF_DIR} (code=exited,
> status=1/FAILURE)
> > > > >
> > > > > Process: 16819
> > > > > ExecStartPre=/usr/share/elasticsearch/bin/
> > > elasticsearch-systemd-pre-exec
> > > > > (code=exited, status=0/SUCCESS)
> > > > >
> > > > > Main PID: 16821 (code=exited, status=1/FAILURE)
> > > > >
> > > > >
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > > loadFromStream(Settings.java:1080)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > > loadFromPath(Settings.java:1067)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.node.internal.InternalSettingsPreparer.
> > > > > prepareEnvironment(InternalSettingsPreparer.java:88)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.bootstrap.Bootstrap.initialSettings(
> > > Bootstrap.java:202)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > at
> > > > > org.elasticsearch.bootstrap.Elasticsearch.main(
> > Elasticsearch.java:35)
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com
> elasticsearch[16821]:
> > > Refer
> > > > > to
> > > > > the log for complete error details.
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > > elasticsearch.service: main process exited, code=exited,
> > > status=1/FAILURE
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> > > > > elasticsearch.service entered failed state.
> > > > >
> > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-
> 04-25%2022:14:58%20EDT
> > >
> > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > > elasticsearch.service failed.
> > > > >
> > > > > INFO 2017-04-26 08:55:19,962
> > > > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
> > > > >
> > > > >
> > > > > On April 26, 2017 at 08:27:27, Otto Fowler (
> ottobackwards@gmail.com)
> > > > > wrote:
> > > > >
> > > > > Can you describe what ES issues you are working on?
> > > > > Fixing the repos got everything installed, but my ES components
> don’t
> > > > stay
> > > > > running.
> > > > > I need to harvest the errors.
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 16:46:00, Otto Fowler (
> ottobackwards@gmail.com)
> > > > > wrote:
> > > > >
> > > > > Nm. sorry. I fixed it.
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 16:42:05, Otto Fowler (
> ottobackwards@gmail.com)
> > > > > wrote:
> > > > >
> > > > > Ok, now I see the repos in the ‘pick version’ screen, but it is
> > erring
> > > on
> > > > > the f://localrepo
> > > > > even though the folder exists, there is no repodata/repomd.xml.
> > > > >
> > > > > What is the command to create a local repo?
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 16:05:17, Otto Fowler (
> ottobackwards@gmail.com)
> > > > > wrote:
> > > > >
> > > > > I was going by the HW community page.
> > > > >
> > > > > Ok, Let me try it
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com)
> > > wrote:
> > > > >
> > > > > That would do it. It requires 2.4.2+. I would have sworn I put
> that
> > in
> > > > the
> > > > > README, but I must have only annotated the PR. :(
> > > > >
> > > > > I'll get that in asap.
> > > > >
> > > > > -D...
> > > > >
> > > > >
> > > > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <
> > ottobackwards@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > 2.4.1 as a matter of fact
> > > > > >
> > > > > >
> > > > > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com)
>
> > > > wrote:
> > > > > >
> > > > > > Any chance they're running Ambari < 2.4.2?
> > > > > >
> > > > > > -D...
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > > > > > michael.miklavcic@gmail.com> wrote:
> > > > > >
> > > > > > > Hey Otto, I don't have wizard screenshots in front of me right
> > now
> > > to
> > > > > > say
> > > > > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't
> run
> > > > into
> > > > > > > anything like that, but next time I run through the install I
> can
> > > > take
> > > > > > > another look.
> > > > > > >
> > > > > > > https://github.com/apache/incubator-metron/blob/master/
> > > > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > > > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
>
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
> > > > ottobackwards@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > stderr:
> > > > > > > >
> > > > > > > > Traceback (most recent call last):
> > > > > > > >
> > > > > > > > File "/var/lib/ambari-agent/cache/
> > common-services/ELASTICSEARCH/
> > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in
> <module>
> > > > > > > >
> > > > > > > > Elasticsearch().execute()
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > libraries/script/script.py",
> > > > > > > > line 280, in execute
> > > > > > > >
> > > > > > > > method(env)
> > > > > > > >
> > > > > > > > File "/var/lib/ambari-agent/cache/
> > common-services/ELASTICSEARCH/
> > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in
> install
> > > > > > > >
> > > > > > > > self.install_packages(env)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > libraries/script/script.py",
> > > > > > > > line 567, in install_packages
> > > > > > > >
> > > > > > > > retry_count=agent_stack_retry_count)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/base.py",
> > > > > > > > line 155, in __init__
> > > > > > > >
> > > > > > > > self.env.run()
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/environment.py",
> > > > > > > > line 160, in run
> > > > > > > >
> > > > > > > > self.run_action(resource, action)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/environment.py",
> > > > > > > > line 124, in run_action
> > > > > > > >
> > > > > > > > provider_action()
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > > core/providers/package/__init__.py", line 54, in
> > action_install
> > > > > > > >
> > > > > > > > self.install_package(package_name, self.resource.use_repos,
> > > > > > > > self.resource.skip_repos)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > > core/providers/package/yumrpm.py", line 49, in
> install_package
> > > > > > > >
> > > > > > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > > > > > logoutput=self.get_logoutput())
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > > core/providers/package/__init__.py", line 83, in
> > > > > > > checked_call_with_retries
> > > > > > > >
> > > > > > > > return self._call_with_retries(cmd, is_checked=True,
> **kwargs)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > > core/providers/package/__init__.py", line 91, in
> > > _call_with_retries
> > > > > > > >
> > > > > > > > code, out = func(cmd, **kwargs)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/shell.py",
> > > > > > > > line 71, in inner
> > > > > > > >
> > > > > > > > result = function(command, **kwargs)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/shell.py",
> > > > > > > > line 93, in checked_call
> > > > > > > >
> > > > > > > > tries=tries, try_sleep=try_sleep)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/shell.py",
> > > > > > > > line 141, in _call_wrapper
> > > > > > > >
> > > > > > > > result = _call(command, **kwargs_copy)
> > > > > > > >
> > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/shell.py",
> > > > > > > > line 294, in _call
> > > > > > > >
> > > > > > > > raise Fail(err_msg)
> > > > > > > >
> > > > > > > > resource_management.core.exceptions.Fail: Execution of
> > > > '/usr/bin/yum
> > > > > > -d
> > > > > > > 0
> > > > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error:
> Nothing
> > > to
> > > > do
> > > > > > > >
> > > > > > > > stdout:
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,669
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Using
> > > > hadoop
> > > > > > > conf
> > > > > > > > dir: /usr/hdp/current/hadoop-client/conf
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,671
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['metron']
> > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,673
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['metron']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,740
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['livy']
> > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,741
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['livy']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,774
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > > Group['elasticsearch'] {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,775
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['elasticsearch']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,812
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['spark']
> > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,812
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['spark']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,848
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['zeppelin']
> > > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,849
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['zeppelin']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,884
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['hadoop']
> > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,885
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['hadoop']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,921
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['kibana']
> > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,922
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['kibana']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,959
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['users']
> > > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,960
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > User['hive']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:48,961
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Adding
> > > > user
> > > > > > > > User['hive']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,227
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['storm']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,227
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['storm']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,388
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['zookeeper']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,388
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['zookeeper']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,518
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > > User['infra-solr'] {'gid': 'hadoop',
> 'fetch_nonlocal_groups':
> > > True,
> > > > > > > > 'groups': [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,518
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['infra-solr']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,578
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['ams']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,578
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['ams']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,625
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['tez']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'users']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,626
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['tez']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,660
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['zeppelin']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,660
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['zeppelin']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,705
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['metron']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,705
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['metron']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,760
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['livy']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,760
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['livy']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,803
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > > User['elasticsearch'] {'gid': 'hadoop',
> > 'fetch_nonlocal_groups':
> > > > > True,
> > > > > > > > 'groups': [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,804
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['elasticsearch']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,837
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['spark']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,837
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['spark']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,885
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['ambari-qa']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'users']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,885
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['ambari-qa']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,928
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['flume']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,928
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['flume']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,972
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['kafka']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:49,973
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> Adding
> > > > user
> > > > > > > > User['kafka']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,025
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['hdfs']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,026
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['hdfs']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,075
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['yarn']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,075
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['yarn']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,125
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > User['kibana']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,126
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['kibana']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,175
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > User['mapred']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,176
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['mapred']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,223
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['hbase']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,224
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['hbase']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,271
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['hcat']
> > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > > [u'hadoop']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,272
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > user
> > > > > > > > User['hcat']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,320
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,333
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Writing
> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it
> > > doesn't
> > > > > > exist
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,334
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from
> 644
> > > to
> > > > > 555
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,335
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if':
> > > '(test
> > > > > > $(id
> > > > > > > -u
> > > > > > > > ambari-qa) -gt 1000) || (false)'}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,364
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Skipping
> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to
> not_if
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,365
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
> > > 'create_parents':
> > > > > > True,
> > > > > > > > 'mode': 0775, 'cd_access': 'a'}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,366
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Creating
> > > > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't
> exist.
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,366
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > owner
> > > > > > > > for /tmp/hbase-hbase from 0 to hbase
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,366
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > > > permission for /tmp/hbase-hbase from 755 to 775
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,368
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,369
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > > >
> > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > > hbase-hbase']
> > > > > >
> > > > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,393
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Skipping
> > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > > >
> > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > > hbase-hbase']
> > > > > >
> > > > > > > > due to not_if
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,394
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Group['hdfs']
> > > > > > {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,395
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Adding
> > > > group
> > > > > > > > Group['hdfs']
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,430
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['hdfs']
> > > > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop',
> u'hdfs']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,431
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Modifying
> > > > > > user
> > > > > > > > hdfs
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,474
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS
> > Type:
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,474
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,474
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Creating
> > > > > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,475
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
>
> > > > > > {'owner':
> > > > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,476
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Creating
> > > > > > > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
>
> > > > > tmpdir']
> > > > > >
> > > > > > > > since it doesn't exist.
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,476
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > owner
> > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0
> to
> > > hdfs
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,476
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > group
> > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0
> to
> > > > hadoop
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,477
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Changing
> > > > > > > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
>
> > > from
> > > > > > 755
> > > > > > > > to 1777
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,477
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > > > > > {'create_parents': True}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,478
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Creating
> > > > > > > > directory Directory['/var/lib/ambari-
> > > agent/tmp/AMBARI-artifacts/']
> > > > > > since
> > > > > > > > it doesn't exist.
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,479
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > > > > > {'content':
> > > > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:50,504
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Downloading
> > > > > > the
> > > > > > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > > > > > linux-x64.tar.gz
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:53,793
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > > > Directory['/usr/jdk64'] {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:53,793
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > Creating
> > > > > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:53,794
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > > Execute[('chmod',
> > > > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > > > > > >
> > > > > > > > 2017-04-25 14:12:53,824
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > Execute['cd
> > > > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> > > > > ambari-sudo.sh
> > > > > > cp
> > > > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64']
> {}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,439
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
> > {'action':
> > > > > > > > ['delete']}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,440
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Removing
> > > > > > > > directory Directory['/var/lib/ambari-
> > agent/tmp/jdk_tmp_1BRMsS']
> > > and
> > > > > > all
> > > > > > > > its content
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,620
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
> > > 'cd_access':
> > > > > > 'a'}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,621
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > Execute[('chmod',
> > > > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,682
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Initializing
> > > > > > 2
> > > > > > > > repositories
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,683
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > > > > > ['create'],
> > > > > > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> > > endif
> > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> 'HDP',
> > > > > > > > 'mirror_list': None}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,709
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content':
> > > InlineTemplate(...)}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,711
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Writing
> > > > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,711
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
> > > > http://public-repo-1.
> > > >
> > > > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7',
> 'action':
> > > > > > ['create'],
> > > > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> > > endif
> > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> > > > 'HDP-UTILS',
> > > > > > > > 'mirror_list': None}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,717
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > > > > > InlineTemplate(...)}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,717
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Writing
> > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't
> > exist
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,718
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > Package['unzip']
> > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:00,846
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Installing
> > > > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:03,413
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > > Package['curl']
> > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:03,429
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > Skipping
> > > > > > > > installation of existing package curl
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:03,430
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > > > > Package['hdp-select'] {'retry_on_repo_unavailability':
> False,
> > > > > > > > 'retry_count': 5}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:03,451
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Installing
> > > > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
> > > hdp-select')
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:04,948
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > Install
> > > ES
> > > > > > Data
> > > > > > > > Node
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:04,952
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_
> > unavailability':
> > > > > > False,
> > > > > > > > 'retry_count': 5}
> > > > > > > >
> > > > > > > > 2017-04-25 14:13:05,086
> > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> -
> > > Installing
> > > > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y
> install
> > > > > > > > elasticsearch-2.3.3')
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Command failed after 1 tries
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
> > > > ottobackwards@gmail.com)
> > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > Yes, I’m going to run through again and get the logs. The
> basic
> > > > issue
> > > > > > > is,
> > > > > > > > no repo tab -> if I should have one,
> > > > > > > > and the ES yum command failing with nothing to do, which
> stops
> > > > > > deployment
> > > > > > > > on all the nodes.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > > > > > michael.miklavcic@gmail.com) wrote:
> > > > > > > >
> > > > > > > > I've been doing an internal hwx deployment (HCP) recently
> for
> > > > testing
> > > > > > > > purposes. I have some ES tweaks to work out, but other than
> > that
> > > it
> > > > > > has
> > > > > > > > been running smoothly. Can you share any log info or
> > screenshots?
> > > > > > > >
> > > > > > > > Mike
> > > > > > > >
> > > > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <
> > ottobackwards@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > > Should I see the repo tab in the ambari wizard for metron
> at
> > > this
> > > > > > > point?
> > > > > > > > > Asking for a friend who can’t get the wizard to work :)
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > > --
> > >
> > > Jon
> > >
> > >
> >
>
>
Re: Ambari Wizard: Repo Tab
Posted by Matt Foley <ma...@apache.org>.
zen_discovery_ping_unicast_hosts, if used, also needs square brackets in the parameter string.
Both these should be fixed to provide the square brackets in the template text (as is done by most other params that need them), but I didn’t include that in METRON-634 because it would muck up current users who HAVE included the square brackets in the parameter string, so a db upgrade script is needed. This puts it in the purview of METRON-865 instead of 634.
--Matt
On 4/26/17, 12:19 PM, "Otto Fowler" <ot...@gmail.com> wrote:
Would i have to change that on Centos 7?
On April 26, 2017 at 15:12:38, Nick Allen (nick@nickallen.org) wrote:
Check on the `network_hosts` setting. I just ran into the same issue. I
had to add brackets around it otherwise the file was not valid YAML and it
would crash trying to load the config file.
network.host: ["_lo:ipv4_","_eth0:ipv4_"]
On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler <ot...@gmail.com>
wrote:
> I think it is literally crashing trying to load the config file
>
>
>
> On April 26, 2017 at 15:00:25, Michael Miklavcic (
> michael.miklavcic@gmail.com) wrote:
>
> A couple more things you should check, given this setup:
>
> - 1 master node (not as data node)
> - 2 data nodes
>
> I'd use something like this:
>
> - gateway_recover_after_data_nodes=1 or 2
> - index_number_of_replicas=0 or 1 (2 data nodes, with 2 total copies of
> the index - if you set this value to 2, it would need 1 more data node
> to
> attain 1 orig index + 2 replicas)
>
>
>
> On Wed, Apr 26, 2017 at 12:02 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > Actually never mind. I am not sure how to read the exception. Is it
> > failing to read the configuration?
> >
> >
> > On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > So the elasticsearch service is disabled. That is the issue. When ES
> > calls get status ( service elasticsearch status ) it fails.
> > I don’t know systemd too well, but does that mean we didn’t do the
equiv
> > of chkconfig on?
> >
> >
> >
> > On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Getting the same error in ES as before from what I can see
> >
> >
> > On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > I still can’t keep ES running, it dies.
> > Also, I’m getting exceptions from enrichment trying to restart when the
> > topology wasn’t running.
> >
> > Is there some coordination in this work? Should I just log jiras?
> > If someone is already sussing out ES, we can sync up
> >
> >
> >
> > On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com)
> wrote:
> >
> > I'm also interested to know why that's important at such a small scale.
> >
> > Jon
> >
> > On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > I am following
> > >
> > > https://community.hortonworks.com/articles/60805/deploying-
> > a-fresh-metron-cluster-using-ambari-serv.html
> > > I DO have master and data node together.
> > > Why is that a problem?
> > >
> > > I will try again with master and data node separate.
> > >
> > >
> > > On April 26, 2017 at 10:41:59, Michael Miklavcic (
> > > michael.miklavcic@gmail.com) wrote:
> > >
> > > Hey Otto,
> > >
> > > How do you have the ES nodes configured? For a base install I would
> > setup 1
> > > master (NOT as data node) and 2 data nodes (NOT on the same node as
the
> > > master). This is the install configuration I got working. You can
also
> > > modify some configuration properties around master node as data node,
> > index
> > > replicas, and gateway recovery to get it working differently, but
this
> is
> > > what will work OOTB with the default config settings from the mpack.
If
> > > you've already setup a master node and a data node on the same host,
> > we'll
> > > need to re-install.
> > >
> > > Mike
> > >
> > >
> > > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <ot...@gmail.com>
> > > wrote:
> > >
> > > > File
> > > > "/usr/lib/python2.6/site-packages/resource_management/
> core/shell.py",
> > > line
> > > > 293, in _call
> > > >
> > > > raise ExecutionFailed(err_msg, code, out, err)
> > > >
> > > > ExecutionFailed: Execution of 'service elasticsearch status'
returned
> > 3.
> > > ●
> > > > elasticsearch.service - Elasticsearch
> > > >
> > > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> > > > disabled;
> > > > vendor preset: disabled)
> > > >
> > > > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58
EDT
> > > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
> > > >
> > > > Docs: http://www.elastic.co
> > > >
> > > > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch
> > > > -Des.pidfile=${PID_DIR}/elasticsearch.pid
> -Des.default.path.home=${ES_
> > > > HOME}
> > > > -Des.default.path.logs=${LOG_DIR}
-Des.default.path.data=${DATA_DIR}
> > > > -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
> > > >
> > > > Process: 16819
> > > > ExecStartPre=/usr/share/elasticsearch/bin/
> > elasticsearch-systemd-pre-exec
> > > > (code=exited, status=0/SUCCESS)
> > > >
> > > > Main PID: 16821 (code=exited, status=1/FAILURE)
> > > >
> > > >
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > loadFromStream(Settings.java:1080)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > loadFromPath(Settings.java:1067)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.node.internal.InternalSettingsPreparer.
> > > > prepareEnvironment(InternalSettingsPreparer.java:88)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.bootstrap.Bootstrap.initialSettings(
> > Bootstrap.java:202)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.bootstrap.Elasticsearch.main(
> Elasticsearch.java:35)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > Refer
> > > > to
> > > > the log for complete error details.
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > elasticsearch.service: main process exited, code=exited,
> > status=1/FAILURE
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> > > > elasticsearch.service entered failed state.
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > elasticsearch.service failed.
> > > >
> > > > INFO 2017-04-26 08:55:19,962
> > > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
> > > >
> > > >
> > > > On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > Can you describe what ES issues you are working on?
> > > > Fixing the repos got everything installed, but my ES components
don’t
> > > stay
> > > > running.
> > > > I need to harvest the errors.
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > Nm. sorry. I fixed it.
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > Ok, now I see the repos in the ‘pick version’ screen, but it is
> erring
> > on
> > > > the f://localrepo
> > > > even though the folder exists, there is no repodata/repomd.xml.
> > > >
> > > > What is the command to create a local repo?
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > I was going by the HW community page.
> > > >
> > > > Ok, Let me try it
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com)
> > wrote:
> > > >
> > > > That would do it. It requires 2.4.2+. I would have sworn I put that
> in
> > > the
> > > > README, but I must have only annotated the PR. :(
> > > >
> > > > I'll get that in asap.
> > > >
> > > > -D...
> > > >
> > > >
> > > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <
> ottobackwards@gmail.com>
> > > > wrote:
> > > >
> > > > > 2.4.1 as a matter of fact
> > > > >
> > > > >
> > > > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com)
> > > wrote:
> > > > >
> > > > > Any chance they're running Ambari < 2.4.2?
> > > > >
> > > > > -D...
> > > > >
> > > > >
> > > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > > > > michael.miklavcic@gmail.com> wrote:
> > > > >
> > > > > > Hey Otto, I don't have wizard screenshots in front of me right
> now
> > to
> > > > > say
> > > > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't
run
> > > into
> > > > > > anything like that, but next time I run through the install I
can
> > > take
> > > > > > another look.
> > > > > >
> > > > > > https://github.com/apache/incubator-metron/blob/master/
> > > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
> > > ottobackwards@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > stderr:
> > > > > > >
> > > > > > > Traceback (most recent call last):
> > > > > > >
> > > > > > > File "/var/lib/ambari-agent/cache/
> common-services/ELASTICSEARCH/
> > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > > > > > >
> > > > > > > Elasticsearch().execute()
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > libraries/script/script.py",
> > > > > > > line 280, in execute
> > > > > > >
> > > > > > > method(env)
> > > > > > >
> > > > > > > File "/var/lib/ambari-agent/cache/
> common-services/ELASTICSEARCH/
> > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > > > > > >
> > > > > > > self.install_packages(env)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > libraries/script/script.py",
> > > > > > > line 567, in install_packages
> > > > > > >
> > > > > > > retry_count=agent_stack_retry_count)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/base.py",
> > > > > > > line 155, in __init__
> > > > > > >
> > > > > > > self.env.run()
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/environment.py",
> > > > > > > line 160, in run
> > > > > > >
> > > > > > > self.run_action(resource, action)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/environment.py",
> > > > > > > line 124, in run_action
> > > > > > >
> > > > > > > provider_action()
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/providers/package/__init__.py", line 54, in
> action_install
> > > > > > >
> > > > > > > self.install_package(package_name, self.resource.use_repos,
> > > > > > > self.resource.skip_repos)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/providers/package/yumrpm.py", line 49, in
install_package
> > > > > > >
> > > > > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > > > > logoutput=self.get_logoutput())
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/providers/package/__init__.py", line 83, in
> > > > > > checked_call_with_retries
> > > > > > >
> > > > > > > return self._call_with_retries(cmd, is_checked=True,
**kwargs)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/providers/package/__init__.py", line 91, in
> > _call_with_retries
> > > > > > >
> > > > > > > code, out = func(cmd, **kwargs)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/shell.py",
> > > > > > > line 71, in inner
> > > > > > >
> > > > > > > result = function(command, **kwargs)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/shell.py",
> > > > > > > line 93, in checked_call
> > > > > > >
> > > > > > > tries=tries, try_sleep=try_sleep)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/shell.py",
> > > > > > > line 141, in _call_wrapper
> > > > > > >
> > > > > > > result = _call(command, **kwargs_copy)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/shell.py",
> > > > > > > line 294, in _call
> > > > > > >
> > > > > > > raise Fail(err_msg)
> > > > > > >
> > > > > > > resource_management.core.exceptions.Fail: Execution of
> > > '/usr/bin/yum
> > > > > -d
> > > > > > 0
> > > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error:
Nothing
> > to
> > > do
> > > > > > >
> > > > > > > stdout:
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,669
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using
> > > hadoop
> > > > > > conf
> > > > > > > dir: /usr/hdp/current/hadoop-client/conf
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,671
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['metron']
> > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,673
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > group
> > > > > > > Group['metron']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,740
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['livy']
> > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,741
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > group
> > > > > > > Group['livy']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,774
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['elasticsearch'] {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,775
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > group
> > > > > > > Group['elasticsearch']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,812
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['spark']
> > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,812
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > group
> > > > > > > Group['spark']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,848
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['zeppelin']
> > > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,849
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > group
> > > > > > > Group['zeppelin']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,884
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['hadoop']
> > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,885
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > group
> > > > > > > Group['hadoop']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,921
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['kibana']
> > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,922
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > group
> > > > > > > Group['kibana']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,959
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['users']
> > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,960
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > User['hive']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,961
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > user
> > > > > > > User['hive']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,227
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['storm']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,227
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['storm']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,388
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['zookeeper']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,388
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['zookeeper']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,518
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> > True,
> > > > > > > 'groups': [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,518
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['infra-solr']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,578
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['ams']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,578
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['ams']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,625
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['tez']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'users']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,626
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['tez']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,660
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['zeppelin']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,660
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['zeppelin']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,705
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['metron']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,705
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['metron']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,760
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['livy']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,760
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['livy']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,803
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['elasticsearch'] {'gid': 'hadoop',
> 'fetch_nonlocal_groups':
> > > > True,
> > > > > > > 'groups': [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,804
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['elasticsearch']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,837
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['spark']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,837
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['spark']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,885
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['ambari-qa']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'users']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,885
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['ambari-qa']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,928
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['flume']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,928
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['flume']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,972
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['kafka']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,973
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['kafka']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,025
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['hdfs']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,026
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Adding
> > > user
> > > > > > > User['hdfs']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,075
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['yarn']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,075
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Adding
> > > user
> > > > > > > User['yarn']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,125
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['kibana']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,126
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Adding
> > > user
> > > > > > > User['kibana']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,175
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['mapred']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,176
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Adding
> > > user
> > > > > > > User['mapred']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,223
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['hbase']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,224
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Adding
> > > user
> > > > > > > User['hbase']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,271
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['hcat']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,272
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Adding
> > > user
> > > > > > > User['hcat']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,320
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,333
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Writing
> > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it
> > doesn't
> > > > > exist
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,334
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from
644
> > to
> > > > 555
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,335
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if':
> > '(test
> > > > > $(id
> > > > > > -u
> > > > > > > ambari-qa) -gt 1000) || (false)'}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,364
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Skipping
> > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,365
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
> > 'create_parents':
> > > > > True,
> > > > > > > 'mode': 0775, 'cd_access': 'a'}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,366
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Creating
> > > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't
exist.
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,366
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > owner
> > > > > > > for /tmp/hbase-hbase from 0 to hbase
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,366
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > > > permission for /tmp/hbase-hbase from 755 to 775
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,368
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,369
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > >
> > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > hbase-hbase']
> > > > >
> > > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,393
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Skipping
> > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > >
> > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > hbase-hbase']
> > > > >
> > > > > > > due to not_if
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,394
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Group['hdfs']
> > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,395
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Adding
> > > group
> > > > > > > Group['hdfs']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,430
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['hdfs']
> > > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop',
u'hdfs']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,431
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Modifying
> > > > > user
> > > > > > > hdfs
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,474
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS
> Type:
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,474
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,474
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Creating
> > > > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,475
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > > > > {'owner':
> > > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,476
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Creating
> > > > > > > directory
Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
> > > > tmpdir']
> > > > >
> > > > > > > since it doesn't exist.
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,476
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > owner
> > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
> > hdfs
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,476
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > group
> > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
> > > hadoop
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,477
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > > > permission for
/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
> > from
> > > > > 755
> > > > > > > to 1777
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,477
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > > > > {'create_parents': True}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,478
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Creating
> > > > > > > directory Directory['/var/lib/ambari-
> > agent/tmp/AMBARI-artifacts/']
> > > > > since
> > > > > > > it doesn't exist.
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,479
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > > > > {'content':
> > > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,504
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Downloading
> > > > > the
> > > > > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > > > > linux-x64.tar.gz
> > > > > > >
> > > > > > > 2017-04-25 14:12:53,793
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > > Directory['/usr/jdk64'] {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:53,793
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> Creating
> > > > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > > > > >
> > > > > > > 2017-04-25 14:12:53,794
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > Execute[('chmod',
> > > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > > > > >
> > > > > > > 2017-04-25 14:12:53,824
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > Execute['cd
> > > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> > > > ambari-sudo.sh
> > > > > cp
> > > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64']
{}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,439
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
> {'action':
> > > > > > > ['delete']}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,440
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Removing
> > > > > > > directory Directory['/var/lib/ambari-
> agent/tmp/jdk_tmp_1BRMsS']
> > and
> > > > > all
> > > > > > > its content
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,620
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
> > 'cd_access':
> > > > > 'a'}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,621
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > Execute[('chmod',
> > > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,682
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Initializing
> > > > > 2
> > > > > > > repositories
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,683
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > > > > ['create'],
> > > > > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> > endif
> > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
'HDP',
> > > > > > > 'mirror_list': None}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,709
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content':
> > InlineTemplate(...)}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,711
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Writing
> > > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,711
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
> > > http://public-repo-1.
> > >
> > > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> > > > > ['create'],
> > > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> > endif
> > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> > > 'HDP-UTILS',
> > > > > > > 'mirror_list': None}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,717
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > > > > InlineTemplate(...)}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,717
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Writing
> > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't
> exist
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,718
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Package['unzip']
> > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,846
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Installing
> > > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > > > > >
> > > > > > > 2017-04-25 14:13:03,413
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > Package['curl']
> > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > >
> > > > > > > 2017-04-25 14:13:03,429
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> Skipping
> > > > > > > installation of existing package curl
> > > > > > >
> > > > > > > 2017-04-25 14:13:03,430
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > > > > > 'retry_count': 5}
> > > > > > >
> > > > > > > 2017-04-25 14:13:03,451
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > Installing
> > > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
> > hdp-select')
> > > > > > >
> > > > > > > 2017-04-25 14:13:04,948
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> Install
> > ES
> > > > > Data
> > > > > > > Node
> > > > > > >
> > > > > > > 2017-04-25 14:13:04,952
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_
> unavailability':
> > > > > False,
> > > > > > > 'retry_count': 5}
> > > > > > >
> > > > > > > 2017-04-25 14:13:05,086
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> -
> > Installing
> > > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y
install
> > > > > > > elasticsearch-2.3.3')
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Command failed after 1 tries
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
> > > ottobackwards@gmail.com)
> > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > Yes, I’m going to run through again and get the logs. The
basic
> > > issue
> > > > > > is,
> > > > > > > no repo tab -> if I should have one,
> > > > > > > and the ES yum command failing with nothing to do, which
stops
> > > > > deployment
> > > > > > > on all the nodes.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > > > > michael.miklavcic@gmail.com) wrote:
> > > > > > >
> > > > > > > I've been doing an internal hwx deployment (HCP) recently for
> > > testing
> > > > > > > purposes. I have some ES tweaks to work out, but other than
> that
> > it
> > > > > has
> > > > > > > been running smoothly. Can you share any log info or
> screenshots?
> > > > > > >
> > > > > > > Mike
> > > > > > >
> > > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <
> ottobackwards@gmail.com
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > > Should I see the repo tab in the ambari wizard for metron
at
> > this
> > > > > > point?
> > > > > > > > Asking for a friend who can’t get the wizard to work :)
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> > --
> >
> > Jon
> >
> >
>
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
Would i have to change that on Centos 7?
On April 26, 2017 at 15:12:38, Nick Allen (nick@nickallen.org) wrote:
Check on the `network_hosts` setting. I just ran into the same issue. I
had to add brackets around it otherwise the file was not valid YAML and it
would crash trying to load the config file.
network.host: ["_lo:ipv4_","_eth0:ipv4_"]
On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler <ot...@gmail.com>
wrote:
> I think it is literally crashing trying to load the config file
>
>
>
> On April 26, 2017 at 15:00:25, Michael Miklavcic (
> michael.miklavcic@gmail.com) wrote:
>
> A couple more things you should check, given this setup:
>
> - 1 master node (not as data node)
> - 2 data nodes
>
> I'd use something like this:
>
> - gateway_recover_after_data_nodes=1 or 2
> - index_number_of_replicas=0 or 1 (2 data nodes, with 2 total copies of
> the index - if you set this value to 2, it would need 1 more data node
> to
> attain 1 orig index + 2 replicas)
>
>
>
> On Wed, Apr 26, 2017 at 12:02 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > Actually never mind. I am not sure how to read the exception. Is it
> > failing to read the configuration?
> >
> >
> > On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > So the elasticsearch service is disabled. That is the issue. When ES
> > calls get status ( service elasticsearch status ) it fails.
> > I don’t know systemd too well, but does that mean we didn’t do the
equiv
> > of chkconfig on?
> >
> >
> >
> > On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Getting the same error in ES as before from what I can see
> >
> >
> > On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > I still can’t keep ES running, it dies.
> > Also, I’m getting exceptions from enrichment trying to restart when the
> > topology wasn’t running.
> >
> > Is there some coordination in this work? Should I just log jiras?
> > If someone is already sussing out ES, we can sync up
> >
> >
> >
> > On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com)
> wrote:
> >
> > I'm also interested to know why that's important at such a small scale.
> >
> > Jon
> >
> > On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > I am following
> > >
> > > https://community.hortonworks.com/articles/60805/deploying-
> > a-fresh-metron-cluster-using-ambari-serv.html
> > > I DO have master and data node together.
> > > Why is that a problem?
> > >
> > > I will try again with master and data node separate.
> > >
> > >
> > > On April 26, 2017 at 10:41:59, Michael Miklavcic (
> > > michael.miklavcic@gmail.com) wrote:
> > >
> > > Hey Otto,
> > >
> > > How do you have the ES nodes configured? For a base install I would
> > setup 1
> > > master (NOT as data node) and 2 data nodes (NOT on the same node as
the
> > > master). This is the install configuration I got working. You can
also
> > > modify some configuration properties around master node as data node,
> > index
> > > replicas, and gateway recovery to get it working differently, but
this
> is
> > > what will work OOTB with the default config settings from the mpack.
If
> > > you've already setup a master node and a data node on the same host,
> > we'll
> > > need to re-install.
> > >
> > > Mike
> > >
> > >
> > > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <ot...@gmail.com>
> > > wrote:
> > >
> > > > File
> > > > "/usr/lib/python2.6/site-packages/resource_management/
> core/shell.py",
> > > line
> > > > 293, in _call
> > > >
> > > > raise ExecutionFailed(err_msg, code, out, err)
> > > >
> > > > ExecutionFailed: Execution of 'service elasticsearch status'
returned
> > 3.
> > > ●
> > > > elasticsearch.service - Elasticsearch
> > > >
> > > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> > > > disabled;
> > > > vendor preset: disabled)
> > > >
> > > > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58
EDT
> > > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
> > > >
> > > > Docs: http://www.elastic.co
> > > >
> > > > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch
> > > > -Des.pidfile=${PID_DIR}/elasticsearch.pid
> -Des.default.path.home=${ES_
> > > > HOME}
> > > > -Des.default.path.logs=${LOG_DIR}
-Des.default.path.data=${DATA_DIR}
> > > > -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
> > > >
> > > > Process: 16819
> > > > ExecStartPre=/usr/share/elasticsearch/bin/
> > elasticsearch-systemd-pre-exec
> > > > (code=exited, status=0/SUCCESS)
> > > >
> > > > Main PID: 16821 (code=exited, status=1/FAILURE)
> > > >
> > > >
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > loadFromStream(Settings.java:1080)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > loadFromPath(Settings.java:1067)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.node.internal.InternalSettingsPreparer.
> > > > prepareEnvironment(InternalSettingsPreparer.java:88)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.bootstrap.Bootstrap.initialSettings(
> > Bootstrap.java:202)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.bootstrap.Elasticsearch.main(
> Elasticsearch.java:35)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > Refer
> > > > to
> > > > the log for complete error details.
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > elasticsearch.service: main process exited, code=exited,
> > status=1/FAILURE
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> > > > elasticsearch.service entered failed state.
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > elasticsearch.service failed.
> > > >
> > > > INFO 2017-04-26 08:55:19,962
> > > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
> > > >
> > > >
> > > > On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > Can you describe what ES issues you are working on?
> > > > Fixing the repos got everything installed, but my ES components
don’t
> > > stay
> > > > running.
> > > > I need to harvest the errors.
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > Nm. sorry. I fixed it.
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > Ok, now I see the repos in the ‘pick version’ screen, but it is
> erring
> > on
> > > > the f://localrepo
> > > > even though the folder exists, there is no repodata/repomd.xml.
> > > >
> > > > What is the command to create a local repo?
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > I was going by the HW community page.
> > > >
> > > > Ok, Let me try it
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com)
> > wrote:
> > > >
> > > > That would do it. It requires 2.4.2+. I would have sworn I put that
> in
> > > the
> > > > README, but I must have only annotated the PR. :(
> > > >
> > > > I'll get that in asap.
> > > >
> > > > -D...
> > > >
> > > >
> > > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <
> ottobackwards@gmail.com>
> > > > wrote:
> > > >
> > > > > 2.4.1 as a matter of fact
> > > > >
> > > > >
> > > > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com)
> > > wrote:
> > > > >
> > > > > Any chance they're running Ambari < 2.4.2?
> > > > >
> > > > > -D...
> > > > >
> > > > >
> > > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > > > > michael.miklavcic@gmail.com> wrote:
> > > > >
> > > > > > Hey Otto, I don't have wizard screenshots in front of me right
> now
> > to
> > > > > say
> > > > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't
run
> > > into
> > > > > > anything like that, but next time I run through the install I
can
> > > take
> > > > > > another look.
> > > > > >
> > > > > > https://github.com/apache/incubator-metron/blob/master/
> > > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
> > > ottobackwards@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > stderr:
> > > > > > >
> > > > > > > Traceback (most recent call last):
> > > > > > >
> > > > > > > File "/var/lib/ambari-agent/cache/
> common-services/ELASTICSEARCH/
> > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > > > > > >
> > > > > > > Elasticsearch().execute()
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > libraries/script/script.py",
> > > > > > > line 280, in execute
> > > > > > >
> > > > > > > method(env)
> > > > > > >
> > > > > > > File "/var/lib/ambari-agent/cache/
> common-services/ELASTICSEARCH/
> > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > > > > > >
> > > > > > > self.install_packages(env)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > libraries/script/script.py",
> > > > > > > line 567, in install_packages
> > > > > > >
> > > > > > > retry_count=agent_stack_retry_count)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/base.py",
> > > > > > > line 155, in __init__
> > > > > > >
> > > > > > > self.env.run()
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/environment.py",
> > > > > > > line 160, in run
> > > > > > >
> > > > > > > self.run_action(resource, action)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/environment.py",
> > > > > > > line 124, in run_action
> > > > > > >
> > > > > > > provider_action()
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/providers/package/__init__.py", line 54, in
> action_install
> > > > > > >
> > > > > > > self.install_package(package_name, self.resource.use_repos,
> > > > > > > self.resource.skip_repos)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/providers/package/yumrpm.py", line 49, in
install_package
> > > > > > >
> > > > > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > > > > logoutput=self.get_logoutput())
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/providers/package/__init__.py", line 83, in
> > > > > > checked_call_with_retries
> > > > > > >
> > > > > > > return self._call_with_retries(cmd, is_checked=True,
**kwargs)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/providers/package/__init__.py", line 91, in
> > _call_with_retries
> > > > > > >
> > > > > > > code, out = func(cmd, **kwargs)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/shell.py",
> > > > > > > line 71, in inner
> > > > > > >
> > > > > > > result = function(command, **kwargs)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/shell.py",
> > > > > > > line 93, in checked_call
> > > > > > >
> > > > > > > tries=tries, try_sleep=try_sleep)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/shell.py",
> > > > > > > line 141, in _call_wrapper
> > > > > > >
> > > > > > > result = _call(command, **kwargs_copy)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/shell.py",
> > > > > > > line 294, in _call
> > > > > > >
> > > > > > > raise Fail(err_msg)
> > > > > > >
> > > > > > > resource_management.core.exceptions.Fail: Execution of
> > > '/usr/bin/yum
> > > > > -d
> > > > > > 0
> > > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error:
Nothing
> > to
> > > do
> > > > > > >
> > > > > > > stdout:
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,669
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using
> > > hadoop
> > > > > > conf
> > > > > > > dir: /usr/hdp/current/hadoop-client/conf
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,671
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['metron']
> > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,673
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > group
> > > > > > > Group['metron']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,740
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['livy']
> > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,741
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > group
> > > > > > > Group['livy']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,774
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['elasticsearch'] {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,775
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > group
> > > > > > > Group['elasticsearch']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,812
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['spark']
> > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,812
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > group
> > > > > > > Group['spark']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,848
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['zeppelin']
> > > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,849
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > group
> > > > > > > Group['zeppelin']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,884
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['hadoop']
> > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,885
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > group
> > > > > > > Group['hadoop']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,921
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['kibana']
> > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,922
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > group
> > > > > > > Group['kibana']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,959
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['users']
> > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,960
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > User['hive']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,961
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Adding
> > > user
> > > > > > > User['hive']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,227
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['storm']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,227
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['storm']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,388
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['zookeeper']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,388
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['zookeeper']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,518
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> > True,
> > > > > > > 'groups': [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,518
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['infra-solr']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,578
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['ams']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,578
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['ams']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,625
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['tez']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'users']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,626
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['tez']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,660
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['zeppelin']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,660
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['zeppelin']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,705
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['metron']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,705
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['metron']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,760
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['livy']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,760
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['livy']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,803
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['elasticsearch'] {'gid': 'hadoop',
> 'fetch_nonlocal_groups':
> > > > True,
> > > > > > > 'groups': [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,804
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['elasticsearch']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,837
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['spark']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,837
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['spark']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,885
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['ambari-qa']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'users']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,885
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['ambari-qa']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,928
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['flume']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,928
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['flume']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,972
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['kafka']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,973
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
Adding
> > > user
> > > > > > > User['kafka']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,025
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['hdfs']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,026
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Adding
> > > user
> > > > > > > User['hdfs']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,075
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['yarn']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,075
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Adding
> > > user
> > > > > > > User['yarn']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,125
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['kibana']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,126
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Adding
> > > user
> > > > > > > User['kibana']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,175
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['mapred']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,176
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Adding
> > > user
> > > > > > > User['mapred']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,223
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['hbase']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,224
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Adding
> > > user
> > > > > > > User['hbase']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,271
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['hcat']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,272
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Adding
> > > user
> > > > > > > User['hcat']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,320
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,333
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Writing
> > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it
> > doesn't
> > > > > exist
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,334
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from
644
> > to
> > > > 555
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,335
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if':
> > '(test
> > > > > $(id
> > > > > > -u
> > > > > > > ambari-qa) -gt 1000) || (false)'}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,364
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Skipping
> > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,365
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
> > 'create_parents':
> > > > > True,
> > > > > > > 'mode': 0775, 'cd_access': 'a'}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,366
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Creating
> > > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't
exist.
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,366
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > owner
> > > > > > > for /tmp/hbase-hbase from 0 to hbase
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,366
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > > > permission for /tmp/hbase-hbase from 755 to 775
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,368
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,369
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > >
> > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > hbase-hbase']
> > > > >
> > > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,393
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Skipping
> > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > >
> > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > hbase-hbase']
> > > > >
> > > > > > > due to not_if
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,394
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Group['hdfs']
> > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,395
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Adding
> > > group
> > > > > > > Group['hdfs']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,430
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['hdfs']
> > > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop',
u'hdfs']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,431
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Modifying
> > > > > user
> > > > > > > hdfs
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,474
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS
> Type:
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,474
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,474
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Creating
> > > > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,475
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > > > > {'owner':
> > > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,476
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Creating
> > > > > > > directory
Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
> > > > tmpdir']
> > > > >
> > > > > > > since it doesn't exist.
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,476
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > owner
> > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
> > hdfs
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,476
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > group
> > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
> > > hadoop
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,477
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > > > permission for
/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
> > from
> > > > > 755
> > > > > > > to 1777
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,477
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > > > > {'create_parents': True}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,478
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Creating
> > > > > > > directory Directory['/var/lib/ambari-
> > agent/tmp/AMBARI-artifacts/']
> > > > > since
> > > > > > > it doesn't exist.
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,479
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > > > > {'content':
> > > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,504
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Downloading
> > > > > the
> > > > > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > > > > linux-x64.tar.gz
> > > > > > >
> > > > > > > 2017-04-25 14:12:53,793
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > > Directory['/usr/jdk64'] {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:53,793
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> Creating
> > > > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > > > > >
> > > > > > > 2017-04-25 14:12:53,794
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > Execute[('chmod',
> > > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > > > > >
> > > > > > > 2017-04-25 14:12:53,824
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > Execute['cd
> > > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> > > > ambari-sudo.sh
> > > > > cp
> > > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64']
{}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,439
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
> {'action':
> > > > > > > ['delete']}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,440
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Removing
> > > > > > > directory Directory['/var/lib/ambari-
> agent/tmp/jdk_tmp_1BRMsS']
> > and
> > > > > all
> > > > > > > its content
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,620
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
> > 'cd_access':
> > > > > 'a'}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,621
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > Execute[('chmod',
> > > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,682
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Initializing
> > > > > 2
> > > > > > > repositories
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,683
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > > > > ['create'],
> > > > > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> > endif
> > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
'HDP',
> > > > > > > 'mirror_list': None}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,709
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content':
> > InlineTemplate(...)}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,711
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Writing
> > > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,711
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
> > > http://public-repo-1.
> > >
> > > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> > > > > ['create'],
> > > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> > endif
> > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> > > 'HDP-UTILS',
> > > > > > > 'mirror_list': None}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,717
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > > > > InlineTemplate(...)}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,717
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Writing
> > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't
> exist
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,718
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Package['unzip']
> > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,846
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Installing
> > > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > > > > >
> > > > > > > 2017-04-25 14:13:03,413
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > Package['curl']
> > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > >
> > > > > > > 2017-04-25 14:13:03,429
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> Skipping
> > > > > > > installation of existing package curl
> > > > > > >
> > > > > > > 2017-04-25 14:13:03,430
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > > > > > 'retry_count': 5}
> > > > > > >
> > > > > > > 2017-04-25 14:13:03,451
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > Installing
> > > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
> > hdp-select')
> > > > > > >
> > > > > > > 2017-04-25 14:13:04,948
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> Install
> > ES
> > > > > Data
> > > > > > > Node
> > > > > > >
> > > > > > > 2017-04-25 14:13:04,952
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_
> unavailability':
> > > > > False,
> > > > > > > 'retry_count': 5}
> > > > > > >
> > > > > > > 2017-04-25 14:13:05,086
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> -
> > Installing
> > > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y
install
> > > > > > > elasticsearch-2.3.3')
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Command failed after 1 tries
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
> > > ottobackwards@gmail.com)
> > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > Yes, I’m going to run through again and get the logs. The
basic
> > > issue
> > > > > > is,
> > > > > > > no repo tab -> if I should have one,
> > > > > > > and the ES yum command failing with nothing to do, which
stops
> > > > > deployment
> > > > > > > on all the nodes.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > > > > michael.miklavcic@gmail.com) wrote:
> > > > > > >
> > > > > > > I've been doing an internal hwx deployment (HCP) recently for
> > > testing
> > > > > > > purposes. I have some ES tweaks to work out, but other than
> that
> > it
> > > > > has
> > > > > > > been running smoothly. Can you share any log info or
> screenshots?
> > > > > > >
> > > > > > > Mike
> > > > > > >
> > > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <
> ottobackwards@gmail.com
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > > Should I see the repo tab in the ambari wizard for metron
at
> > this
> > > > > > point?
> > > > > > > > Asking for a friend who can’t get the wizard to work :)
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> > --
> >
> > Jon
> >
> >
>
Re: Ambari Wizard: Repo Tab
Posted by Nick Allen <ni...@nickallen.org>.
Check on the `network_hosts` setting. I just ran into the same issue. I
had to add brackets around it otherwise the file was not valid YAML and it
would crash trying to load the config file.
network.host: ["_lo:ipv4_","_eth0:ipv4_"]
On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler <ot...@gmail.com>
wrote:
> I think it is literally crashing trying to load the config file
>
>
>
> On April 26, 2017 at 15:00:25, Michael Miklavcic (
> michael.miklavcic@gmail.com) wrote:
>
> A couple more things you should check, given this setup:
>
> - 1 master node (not as data node)
> - 2 data nodes
>
> I'd use something like this:
>
> - gateway_recover_after_data_nodes=1 or 2
> - index_number_of_replicas=0 or 1 (2 data nodes, with 2 total copies of
> the index - if you set this value to 2, it would need 1 more data node
> to
> attain 1 orig index + 2 replicas)
>
>
>
> On Wed, Apr 26, 2017 at 12:02 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > Actually never mind. I am not sure how to read the exception. Is it
> > failing to read the configuration?
> >
> >
> > On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > So the elasticsearch service is disabled. That is the issue. When ES
> > calls get status ( service elasticsearch status ) it fails.
> > I don’t know systemd too well, but does that mean we didn’t do the equiv
> > of chkconfig on?
> >
> >
> >
> > On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Getting the same error in ES as before from what I can see
> >
> >
> > On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > I still can’t keep ES running, it dies.
> > Also, I’m getting exceptions from enrichment trying to restart when the
> > topology wasn’t running.
> >
> > Is there some coordination in this work? Should I just log jiras?
> > If someone is already sussing out ES, we can sync up
> >
> >
> >
> > On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com)
> wrote:
> >
> > I'm also interested to know why that's important at such a small scale.
> >
> > Jon
> >
> > On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > I am following
> > >
> > > https://community.hortonworks.com/articles/60805/deploying-
> > a-fresh-metron-cluster-using-ambari-serv.html
> > > I DO have master and data node together.
> > > Why is that a problem?
> > >
> > > I will try again with master and data node separate.
> > >
> > >
> > > On April 26, 2017 at 10:41:59, Michael Miklavcic (
> > > michael.miklavcic@gmail.com) wrote:
> > >
> > > Hey Otto,
> > >
> > > How do you have the ES nodes configured? For a base install I would
> > setup 1
> > > master (NOT as data node) and 2 data nodes (NOT on the same node as the
> > > master). This is the install configuration I got working. You can also
> > > modify some configuration properties around master node as data node,
> > index
> > > replicas, and gateway recovery to get it working differently, but this
> is
> > > what will work OOTB with the default config settings from the mpack. If
> > > you've already setup a master node and a data node on the same host,
> > we'll
> > > need to re-install.
> > >
> > > Mike
> > >
> > >
> > > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <ot...@gmail.com>
> > > wrote:
> > >
> > > > File
> > > > "/usr/lib/python2.6/site-packages/resource_management/
> core/shell.py",
> > > line
> > > > 293, in _call
> > > >
> > > > raise ExecutionFailed(err_msg, code, out, err)
> > > >
> > > > ExecutionFailed: Execution of 'service elasticsearch status' returned
> > 3.
> > > ●
> > > > elasticsearch.service - Elasticsearch
> > > >
> > > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> > > > disabled;
> > > > vendor preset: disabled)
> > > >
> > > > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58 EDT
> > > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
> > > >
> > > > Docs: http://www.elastic.co
> > > >
> > > > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch
> > > > -Des.pidfile=${PID_DIR}/elasticsearch.pid
> -Des.default.path.home=${ES_
> > > > HOME}
> > > > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR}
> > > > -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
> > > >
> > > > Process: 16819
> > > > ExecStartPre=/usr/share/elasticsearch/bin/
> > elasticsearch-systemd-pre-exec
> > > > (code=exited, status=0/SUCCESS)
> > > >
> > > > Main PID: 16821 (code=exited, status=1/FAILURE)
> > > >
> > > >
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > loadFromStream(Settings.java:1080)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.common.settings.Settings$Builder.
> > > > loadFromPath(Settings.java:1067)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.node.internal.InternalSettingsPreparer.
> > > > prepareEnvironment(InternalSettingsPreparer.java:88)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.bootstrap.Bootstrap.initialSettings(
> > Bootstrap.java:202)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> at
> > > > org.elasticsearch.bootstrap.Elasticsearch.main(
> Elasticsearch.java:35)
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> > Refer
> > > > to
> > > > the log for complete error details.
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > elasticsearch.service: main process exited, code=exited,
> > status=1/FAILURE
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> > > > elasticsearch.service entered failed state.
> > > >
> > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT
> >
> > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > > elasticsearch.service failed.
> > > >
> > > > INFO 2017-04-26 08:55:19,962
> > > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
> > > >
> > > >
> > > > On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > Can you describe what ES issues you are working on?
> > > > Fixing the repos got everything installed, but my ES components don’t
> > > stay
> > > > running.
> > > > I need to harvest the errors.
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > Nm. sorry. I fixed it.
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > Ok, now I see the repos in the ‘pick version’ screen, but it is
> erring
> > on
> > > > the f://localrepo
> > > > even though the folder exists, there is no repodata/repomd.xml.
> > > >
> > > > What is the command to create a local repo?
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > I was going by the HW community page.
> > > >
> > > > Ok, Let me try it
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com)
> > wrote:
> > > >
> > > > That would do it. It requires 2.4.2+. I would have sworn I put that
> in
> > > the
> > > > README, but I must have only annotated the PR. :(
> > > >
> > > > I'll get that in asap.
> > > >
> > > > -D...
> > > >
> > > >
> > > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <
> ottobackwards@gmail.com>
> > > > wrote:
> > > >
> > > > > 2.4.1 as a matter of fact
> > > > >
> > > > >
> > > > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com)
> > > wrote:
> > > > >
> > > > > Any chance they're running Ambari < 2.4.2?
> > > > >
> > > > > -D...
> > > > >
> > > > >
> > > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > > > > michael.miklavcic@gmail.com> wrote:
> > > > >
> > > > > > Hey Otto, I don't have wizard screenshots in front of me right
> now
> > to
> > > > > say
> > > > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't run
> > > into
> > > > > > anything like that, but next time I run through the install I can
> > > take
> > > > > > another look.
> > > > > >
> > > > > > https://github.com/apache/incubator-metron/blob/master/
> > > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
> > > ottobackwards@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > stderr:
> > > > > > >
> > > > > > > Traceback (most recent call last):
> > > > > > >
> > > > > > > File "/var/lib/ambari-agent/cache/
> common-services/ELASTICSEARCH/
> > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > > > > > >
> > > > > > > Elasticsearch().execute()
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > libraries/script/script.py",
> > > > > > > line 280, in execute
> > > > > > >
> > > > > > > method(env)
> > > > > > >
> > > > > > > File "/var/lib/ambari-agent/cache/
> common-services/ELASTICSEARCH/
> > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > > > > > >
> > > > > > > self.install_packages(env)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > libraries/script/script.py",
> > > > > > > line 567, in install_packages
> > > > > > >
> > > > > > > retry_count=agent_stack_retry_count)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/base.py",
> > > > > > > line 155, in __init__
> > > > > > >
> > > > > > > self.env.run()
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/environment.py",
> > > > > > > line 160, in run
> > > > > > >
> > > > > > > self.run_action(resource, action)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/environment.py",
> > > > > > > line 124, in run_action
> > > > > > >
> > > > > > > provider_action()
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/providers/package/__init__.py", line 54, in
> action_install
> > > > > > >
> > > > > > > self.install_package(package_name, self.resource.use_repos,
> > > > > > > self.resource.skip_repos)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/providers/package/yumrpm.py", line 49, in install_package
> > > > > > >
> > > > > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > > > > logoutput=self.get_logoutput())
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/providers/package/__init__.py", line 83, in
> > > > > > checked_call_with_retries
> > > > > > >
> > > > > > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > > core/providers/package/__init__.py", line 91, in
> > _call_with_retries
> > > > > > >
> > > > > > > code, out = func(cmd, **kwargs)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/shell.py",
> > > > > > > line 71, in inner
> > > > > > >
> > > > > > > result = function(command, **kwargs)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/shell.py",
> > > > > > > line 93, in checked_call
> > > > > > >
> > > > > > > tries=tries, try_sleep=try_sleep)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/shell.py",
> > > > > > > line 141, in _call_wrapper
> > > > > > >
> > > > > > > result = _call(command, **kwargs_copy)
> > > > > > >
> > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/shell.py",
> > > > > > > line 294, in _call
> > > > > > >
> > > > > > > raise Fail(err_msg)
> > > > > > >
> > > > > > > resource_management.core.exceptions.Fail: Execution of
> > > '/usr/bin/yum
> > > > > -d
> > > > > > 0
> > > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing
> > to
> > > do
> > > > > > >
> > > > > > > stdout:
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,669
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using
> > > hadoop
> > > > > > conf
> > > > > > > dir: /usr/hdp/current/hadoop-client/conf
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,671
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['metron']
> > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,673
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > > group
> > > > > > > Group['metron']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,740
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['livy']
> > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,741
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > > group
> > > > > > > Group['livy']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,774
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > > Group['elasticsearch'] {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,775
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > > group
> > > > > > > Group['elasticsearch']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,812
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['spark']
> > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,812
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > > group
> > > > > > > Group['spark']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,848
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['zeppelin']
> > > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,849
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > > group
> > > > > > > Group['zeppelin']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,884
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['hadoop']
> > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,885
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > > group
> > > > > > > Group['hadoop']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,921
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['kibana']
> > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,922
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > > group
> > > > > > > Group['kibana']
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,959
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['users']
> > > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,960
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > User['hive']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:48,961
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > > user
> > > > > > > User['hive']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,227
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['storm']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,227
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > > user
> > > > > > > User['storm']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,388
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['zookeeper']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,388
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > > user
> > > > > > > User['zookeeper']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,518
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> > True,
> > > > > > > 'groups': [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,518
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > > user
> > > > > > > User['infra-solr']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,578
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['ams']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,578
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > > user
> > > > > > > User['ams']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,625
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['tez']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'users']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,626
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > > user
> > > > > > > User['tez']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,660
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['zeppelin']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,660
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > > user
> > > > > > > User['zeppelin']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,705
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['metron']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,705
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > > user
> > > > > > > User['metron']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,760
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['livy']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,760
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > > user
> > > > > > > User['livy']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,803
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > > User['elasticsearch'] {'gid': 'hadoop',
> 'fetch_nonlocal_groups':
> > > > True,
> > > > > > > 'groups': [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,804
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > > user
> > > > > > > User['elasticsearch']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,837
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['spark']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,837
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > > user
> > > > > > > User['spark']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,885
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['ambari-qa']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'users']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,885
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > > user
> > > > > > > User['ambari-qa']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,928
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['flume']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,928
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > > user
> > > > > > > User['flume']
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,972
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['kafka']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:49,973
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > > user
> > > > > > > User['kafka']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,025
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['hdfs']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,026
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > > user
> > > > > > > User['hdfs']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,075
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['yarn']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,075
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > > user
> > > > > > > User['yarn']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,125
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['kibana']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,126
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > > user
> > > > > > > User['kibana']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,175
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > User['mapred']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,176
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > > user
> > > > > > > User['mapred']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,223
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['hbase']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,224
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > > user
> > > > > > > User['hbase']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,271
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['hcat']
> > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > > [u'hadoop']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,272
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > > user
> > > > > > > User['hcat']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,320
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,333
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Writing
> > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it
> > doesn't
> > > > > exist
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,334
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644
> > to
> > > > 555
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,335
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if':
> > '(test
> > > > > $(id
> > > > > > -u
> > > > > > > ambari-qa) -gt 1000) || (false)'}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,364
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Skipping
> > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,365
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
> > 'create_parents':
> > > > > True,
> > > > > > > 'mode': 0775, 'cd_access': 'a'}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,366
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Creating
> > > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,366
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > owner
> > > > > > > for /tmp/hbase-hbase from 0 to hbase
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,366
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > > > permission for /tmp/hbase-hbase from 755 to 775
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,368
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,369
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > >
> > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > hbase-hbase']
> > > > >
> > > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,393
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Skipping
> > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > > >
> > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> > hbase-hbase']
> > > > >
> > > > > > > due to not_if
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,394
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Group['hdfs']
> > > > > {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,395
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > > group
> > > > > > > Group['hdfs']
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,430
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['hdfs']
> > > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,431
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Modifying
> > > > > user
> > > > > > > hdfs
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,474
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS
> Type:
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,474
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,474
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Creating
> > > > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,475
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > > > > {'owner':
> > > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,476
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Creating
> > > > > > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
> > > > tmpdir']
> > > > >
> > > > > > > since it doesn't exist.
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,476
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > owner
> > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
> > hdfs
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,476
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > group
> > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
> > > hadoop
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,477
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Changing
> > > > > > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
> > from
> > > > > 755
> > > > > > > to 1777
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,477
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > > > > {'create_parents': True}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,478
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Creating
> > > > > > > directory Directory['/var/lib/ambari-
> > agent/tmp/AMBARI-artifacts/']
> > > > > since
> > > > > > > it doesn't exist.
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,479
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > > > > {'content':
> > > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > > > > >
> > > > > > > 2017-04-25 14:12:50,504
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Downloading
> > > > > the
> > > > > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > > > > linux-x64.tar.gz
> > > > > > >
> > > > > > > 2017-04-25 14:12:53,793
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > > Directory['/usr/jdk64'] {}
> > > > > > >
> > > > > > > 2017-04-25 14:12:53,793
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> Creating
> > > > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > > > > >
> > > > > > > 2017-04-25 14:12:53,794
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > Execute[('chmod',
> > > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > > > > >
> > > > > > > 2017-04-25 14:12:53,824
> > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > Execute['cd
> > > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> > > > ambari-sudo.sh
> > > > > cp
> > > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,439
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
> {'action':
> > > > > > > ['delete']}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,440
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Removing
> > > > > > > directory Directory['/var/lib/ambari-
> agent/tmp/jdk_tmp_1BRMsS']
> > and
> > > > > all
> > > > > > > its content
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,620
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
> > 'cd_access':
> > > > > 'a'}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,621
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > Execute[('chmod',
> > > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,682
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Initializing
> > > > > 2
> > > > > > > repositories
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,683
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > > > > ['create'],
> > > > > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> > endif
> > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > > > > > 'mirror_list': None}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,709
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content':
> > InlineTemplate(...)}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,711
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Writing
> > > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,711
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
> > > http://public-repo-1.
> > >
> > > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> > > > > ['create'],
> > > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> > endif
> > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> > > 'HDP-UTILS',
> > > > > > > 'mirror_list': None}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,717
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > > > > InlineTemplate(...)}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,717
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Writing
> > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't
> exist
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,718
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Package['unzip']
> > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > >
> > > > > > > 2017-04-25 14:13:00,846
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Installing
> > > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > > > > >
> > > > > > > 2017-04-25 14:13:03,413
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > Package['curl']
> > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > > >
> > > > > > > 2017-04-25 14:13:03,429
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> Skipping
> > > > > > > installation of existing package curl
> > > > > > >
> > > > > > > 2017-04-25 14:13:03,430
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > > > > > 'retry_count': 5}
> > > > > > >
> > > > > > > 2017-04-25 14:13:03,451
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > Installing
> > > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
> > hdp-select')
> > > > > > >
> > > > > > > 2017-04-25 14:13:04,948
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> Install
> > ES
> > > > > Data
> > > > > > > Node
> > > > > > >
> > > > > > > 2017-04-25 14:13:04,952
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_
> unavailability':
> > > > > False,
> > > > > > > 'retry_count': 5}
> > > > > > >
> > > > > > > 2017-04-25 14:13:05,086
> > > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> -
> > Installing
> > > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > > > > > elasticsearch-2.3.3')
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Command failed after 1 tries
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
> > > ottobackwards@gmail.com)
> > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > Yes, I’m going to run through again and get the logs. The basic
> > > issue
> > > > > > is,
> > > > > > > no repo tab -> if I should have one,
> > > > > > > and the ES yum command failing with nothing to do, which stops
> > > > > deployment
> > > > > > > on all the nodes.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > > > > michael.miklavcic@gmail.com) wrote:
> > > > > > >
> > > > > > > I've been doing an internal hwx deployment (HCP) recently for
> > > testing
> > > > > > > purposes. I have some ES tweaks to work out, but other than
> that
> > it
> > > > > has
> > > > > > > been running smoothly. Can you share any log info or
> screenshots?
> > > > > > >
> > > > > > > Mike
> > > > > > >
> > > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <
> ottobackwards@gmail.com
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > > Should I see the repo tab in the ambari wizard for metron at
> > this
> > > > > > point?
> > > > > > > > Asking for a friend who can’t get the wizard to work :)
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> > --
> >
> > Jon
> >
> >
>
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
I think it is literally crashing trying to load the config file
On April 26, 2017 at 15:00:25, Michael Miklavcic (
michael.miklavcic@gmail.com) wrote:
A couple more things you should check, given this setup:
- 1 master node (not as data node)
- 2 data nodes
I'd use something like this:
- gateway_recover_after_data_nodes=1 or 2
- index_number_of_replicas=0 or 1 (2 data nodes, with 2 total copies of
the index - if you set this value to 2, it would need 1 more data node to
attain 1 orig index + 2 replicas)
On Wed, Apr 26, 2017 at 12:02 PM, Otto Fowler <ot...@gmail.com>
wrote:
> Actually never mind. I am not sure how to read the exception. Is it
> failing to read the configuration?
>
>
> On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> So the elasticsearch service is disabled. That is the issue. When ES
> calls get status ( service elasticsearch status ) it fails.
> I don’t know systemd too well, but does that mean we didn’t do the equiv
> of chkconfig on?
>
>
>
> On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> Getting the same error in ES as before from what I can see
>
>
> On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> I still can’t keep ES running, it dies.
> Also, I’m getting exceptions from enrichment trying to restart when the
> topology wasn’t running.
>
> Is there some coordination in this work? Should I just log jiras?
> If someone is already sussing out ES, we can sync up
>
>
>
> On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com) wrote:
>
> I'm also interested to know why that's important at such a small scale.
>
> Jon
>
> On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com>
> wrote:
>
> > I am following
> >
> > https://community.hortonworks.com/articles/60805/deploying-
> a-fresh-metron-cluster-using-ambari-serv.html
> > I DO have master and data node together.
> > Why is that a problem?
> >
> > I will try again with master and data node separate.
> >
> >
> > On April 26, 2017 at 10:41:59, Michael Miklavcic (
> > michael.miklavcic@gmail.com) wrote:
> >
> > Hey Otto,
> >
> > How do you have the ES nodes configured? For a base install I would
> setup 1
> > master (NOT as data node) and 2 data nodes (NOT on the same node as the
> > master). This is the install configuration I got working. You can also
> > modify some configuration properties around master node as data node,
> index
> > replicas, and gateway recovery to get it working differently, but this is
> > what will work OOTB with the default config settings from the mpack. If
> > you've already setup a master node and a data node on the same host,
> we'll
> > need to re-install.
> >
> > Mike
> >
> >
> > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > File
> > > "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> > line
> > > 293, in _call
> > >
> > > raise ExecutionFailed(err_msg, code, out, err)
> > >
> > > ExecutionFailed: Execution of 'service elasticsearch status' returned
> 3.
> > ●
> > > elasticsearch.service - Elasticsearch
> > >
> > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> > > disabled;
> > > vendor preset: disabled)
> > >
> > > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58 EDT
> > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
> > >
> > > Docs: http://www.elastic.co
> > >
> > > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch
> > > -Des.pidfile=${PID_DIR}/elasticsearch.pid -Des.default.path.home=${ES_
> > > HOME}
> > > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR}
> > > -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
> > >
> > > Process: 16819
> > > ExecStartPre=/usr/share/elasticsearch/bin/
> elasticsearch-systemd-pre-exec
> > > (code=exited, status=0/SUCCESS)
> > >
> > > Main PID: 16821 (code=exited, status=1/FAILURE)
> > >
> > >
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > > org.elasticsearch.common.settings.Settings$Builder.
> > > loadFromStream(Settings.java:1080)
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > > org.elasticsearch.common.settings.Settings$Builder.
> > > loadFromPath(Settings.java:1067)
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > > org.elasticsearch.node.internal.InternalSettingsPreparer.
> > > prepareEnvironment(InternalSettingsPreparer.java:88)
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > > org.elasticsearch.bootstrap.Bootstrap.initialSettings(
> Bootstrap.java:202)
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > > org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> Refer
> > > to
> > > the log for complete error details.
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > elasticsearch.service: main process exited, code=exited,
> status=1/FAILURE
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> > > elasticsearch.service entered failed state.
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > elasticsearch.service failed.
> > >
> > > INFO 2017-04-26 08:55:19,962
> > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
> > >
> > >
> > > On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > Can you describe what ES issues you are working on?
> > > Fixing the repos got everything installed, but my ES components don’t
> > stay
> > > running.
> > > I need to harvest the errors.
> > >
> > >
> > >
> > > On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > Nm. sorry. I fixed it.
> > >
> > >
> > >
> > > On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > Ok, now I see the repos in the ‘pick version’ screen, but it is erring
> on
> > > the f://localrepo
> > > even though the folder exists, there is no repodata/repomd.xml.
> > >
> > > What is the command to create a local repo?
> > >
> > >
> > >
> > > On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > I was going by the HW community page.
> > >
> > > Ok, Let me try it
> > >
> > >
> > >
> > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com)
> wrote:
> > >
> > > That would do it. It requires 2.4.2+. I would have sworn I put that in
> > the
> > > README, but I must have only annotated the PR. :(
> > >
> > > I'll get that in asap.
> > >
> > > -D...
> > >
> > >
> > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
> > > wrote:
> > >
> > > > 2.4.1 as a matter of fact
> > > >
> > > >
> > > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com)
> > wrote:
> > > >
> > > > Any chance they're running Ambari < 2.4.2?
> > > >
> > > > -D...
> > > >
> > > >
> > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > > > michael.miklavcic@gmail.com> wrote:
> > > >
> > > > > Hey Otto, I don't have wizard screenshots in front of me right now
> to
> > > > say
> > > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't run
> > into
> > > > > anything like that, but next time I run through the install I can
> > take
> > > > > another look.
> > > > >
> > > > > https://github.com/apache/incubator-metron/blob/master/
> > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> > > > >
> > > > >
> > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
> > ottobackwards@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > stderr:
> > > > > >
> > > > > > Traceback (most recent call last):
> > > > > >
> > > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > > > > >
> > > > > > Elasticsearch().execute()
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > libraries/script/script.py",
> > > > > > line 280, in execute
> > > > > >
> > > > > > method(env)
> > > > > >
> > > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > > > > >
> > > > > > self.install_packages(env)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > libraries/script/script.py",
> > > > > > line 567, in install_packages
> > > > > >
> > > > > > retry_count=agent_stack_retry_count)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/base.py",
> > > > > > line 155, in __init__
> > > > > >
> > > > > > self.env.run()
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/environment.py",
> > > > > > line 160, in run
> > > > > >
> > > > > > self.run_action(resource, action)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/environment.py",
> > > > > > line 124, in run_action
> > > > > >
> > > > > > provider_action()
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/providers/package/__init__.py", line 54, in action_install
> > > > > >
> > > > > > self.install_package(package_name, self.resource.use_repos,
> > > > > > self.resource.skip_repos)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/providers/package/yumrpm.py", line 49, in install_package
> > > > > >
> > > > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > > > logoutput=self.get_logoutput())
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/providers/package/__init__.py", line 83, in
> > > > > checked_call_with_retries
> > > > > >
> > > > > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/providers/package/__init__.py", line 91, in
> _call_with_retries
> > > > > >
> > > > > > code, out = func(cmd, **kwargs)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/shell.py",
> > > > > > line 71, in inner
> > > > > >
> > > > > > result = function(command, **kwargs)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/shell.py",
> > > > > > line 93, in checked_call
> > > > > >
> > > > > > tries=tries, try_sleep=try_sleep)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/shell.py",
> > > > > > line 141, in _call_wrapper
> > > > > >
> > > > > > result = _call(command, **kwargs_copy)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/shell.py",
> > > > > > line 294, in _call
> > > > > >
> > > > > > raise Fail(err_msg)
> > > > > >
> > > > > > resource_management.core.exceptions.Fail: Execution of
> > '/usr/bin/yum
> > > > -d
> > > > > 0
> > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing
> to
> > do
> > > > > >
> > > > > > stdout:
> > > > > >
> > > > > > 2017-04-25 14:12:48,669
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using
> > hadoop
> > > > > conf
> > > > > > dir: /usr/hdp/current/hadoop-client/conf
> > > > > >
> > > > > > 2017-04-25 14:12:48,671
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['metron']
> > > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,673
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > group
> > > > > > Group['metron']
> > > > > >
> > > > > > 2017-04-25 14:12:48,740
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['livy']
> > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,741
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > group
> > > > > > Group['livy']
> > > > > >
> > > > > > 2017-04-25 14:12:48,774
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['elasticsearch'] {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,775
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > group
> > > > > > Group['elasticsearch']
> > > > > >
> > > > > > 2017-04-25 14:12:48,812
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['spark']
> > > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,812
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > group
> > > > > > Group['spark']
> > > > > >
> > > > > > 2017-04-25 14:12:48,848
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['zeppelin']
> > > > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,849
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > group
> > > > > > Group['zeppelin']
> > > > > >
> > > > > > 2017-04-25 14:12:48,884
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['hadoop']
> > > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,885
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > group
> > > > > > Group['hadoop']
> > > > > >
> > > > > > 2017-04-25 14:12:48,921
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['kibana']
> > > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,922
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > group
> > > > > > Group['kibana']
> > > > > >
> > > > > > 2017-04-25 14:12:48,959
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['users']
> > > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,960
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > User['hive']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:48,961
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > user
> > > > > > User['hive']
> > > > > >
> > > > > > 2017-04-25 14:12:49,227
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['storm']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,227
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['storm']
> > > > > >
> > > > > > 2017-04-25 14:12:49,388
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['zookeeper']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,388
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['zookeeper']
> > > > > >
> > > > > > 2017-04-25 14:12:49,518
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> True,
> > > > > > 'groups': [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,518
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['infra-solr']
> > > > > >
> > > > > > 2017-04-25 14:12:49,578
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['ams']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,578
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['ams']
> > > > > >
> > > > > > 2017-04-25 14:12:49,625
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['tez']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'users']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,626
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['tez']
> > > > > >
> > > > > > 2017-04-25 14:12:49,660
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['zeppelin']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,660
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['zeppelin']
> > > > > >
> > > > > > 2017-04-25 14:12:49,705
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['metron']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,705
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['metron']
> > > > > >
> > > > > > 2017-04-25 14:12:49,760
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['livy']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,760
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['livy']
> > > > > >
> > > > > > 2017-04-25 14:12:49,803
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> > > True,
> > > > > > 'groups': [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,804
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['elasticsearch']
> > > > > >
> > > > > > 2017-04-25 14:12:49,837
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['spark']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,837
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['spark']
> > > > > >
> > > > > > 2017-04-25 14:12:49,885
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['ambari-qa']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'users']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,885
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['ambari-qa']
> > > > > >
> > > > > > 2017-04-25 14:12:49,928
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['flume']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,928
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['flume']
> > > > > >
> > > > > > 2017-04-25 14:12:49,972
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['kafka']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,973
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['kafka']
> > > > > >
> > > > > > 2017-04-25 14:12:50,025
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['hdfs']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:50,026
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > user
> > > > > > User['hdfs']
> > > > > >
> > > > > > 2017-04-25 14:12:50,075
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['yarn']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:50,075
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > user
> > > > > > User['yarn']
> > > > > >
> > > > > > 2017-04-25 14:12:50,125
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['kibana']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:50,126
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > user
> > > > > > User['kibana']
> > > > > >
> > > > > > 2017-04-25 14:12:50,175
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['mapred']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:50,176
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > user
> > > > > > User['mapred']
> > > > > >
> > > > > > 2017-04-25 14:12:50,223
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['hbase']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:50,224
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > user
> > > > > > User['hbase']
> > > > > >
> > > > > > 2017-04-25 14:12:50,271
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['hcat']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:50,272
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > user
> > > > > > User['hcat']
> > > > > >
> > > > > > 2017-04-25 14:12:50,320
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > >
> > > > > > 2017-04-25 14:12:50,333
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it
> doesn't
> > > > exist
> > > > > >
> > > > > > 2017-04-25 14:12:50,334
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644
> to
> > > 555
> > > > > >
> > > > > > 2017-04-25 14:12:50,335
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if':
> '(test
> > > > $(id
> > > > > -u
> > > > > > ambari-qa) -gt 1000) || (false)'}
> > > > > >
> > > > > > 2017-04-25 14:12:50,364
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > > > > >
> > > > > > 2017-04-25 14:12:50,365
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
> 'create_parents':
> > > > True,
> > > > > > 'mode': 0775, 'cd_access': 'a'}
> > > > > >
> > > > > > 2017-04-25 14:12:50,366
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > > > > >
> > > > > > 2017-04-25 14:12:50,366
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > owner
> > > > > > for /tmp/hbase-hbase from 0 to hbase
> > > > > >
> > > > > > 2017-04-25 14:12:50,366
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > > permission for /tmp/hbase-hbase from 755 to 775
> > > > > >
> > > > > > 2017-04-25 14:12:50,368
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > >
> > > > > > 2017-04-25 14:12:50,369
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > >
> > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> hbase-hbase']
> > > >
> > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > > > >
> > > > > > 2017-04-25 14:12:50,393
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > >
> > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> hbase-hbase']
> > > >
> > > > > > due to not_if
> > > > > >
> > > > > > 2017-04-25 14:12:50,394
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Group['hdfs']
> > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:50,395
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > group
> > > > > > Group['hdfs']
> > > > > >
> > > > > > 2017-04-25 14:12:50,430
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['hdfs']
> > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > > > > >
> > > > > > 2017-04-25 14:12:50,431
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Modifying
> > > > user
> > > > > > hdfs
> > > > > >
> > > > > > 2017-04-25 14:12:50,474
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > > > > >
> > > > > > 2017-04-25 14:12:50,474
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > > > >
> > > > > > 2017-04-25 14:12:50,474
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > > > >
> > > > > > 2017-04-25 14:12:50,475
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > > > {'owner':
> > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > > > >
> > > > > > 2017-04-25 14:12:50,476
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
> > > tmpdir']
> > > >
> > > > > > since it doesn't exist.
> > > > > >
> > > > > > 2017-04-25 14:12:50,476
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > owner
> > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
> hdfs
> > > > > >
> > > > > > 2017-04-25 14:12:50,476
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > group
> > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
> > hadoop
> > > > > >
> > > > > > 2017-04-25 14:12:50,477
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
> from
> > > > 755
> > > > > > to 1777
> > > > > >
> > > > > > 2017-04-25 14:12:50,477
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > > > {'create_parents': True}
> > > > > >
> > > > > > 2017-04-25 14:12:50,478
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > > directory Directory['/var/lib/ambari-
> agent/tmp/AMBARI-artifacts/']
> > > > since
> > > > > > it doesn't exist.
> > > > > >
> > > > > > 2017-04-25 14:12:50,479
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > > > {'content':
> > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > > > >
> > > > > > 2017-04-25 14:12:50,504
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Downloading
> > > > the
> > > > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > > > linux-x64.tar.gz
> > > > > >
> > > > > > 2017-04-25 14:12:53,793
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > Directory['/usr/jdk64'] {}
> > > > > >
> > > > > > 2017-04-25 14:12:53,793
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > > > >
> > > > > > 2017-04-25 14:12:53,794
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > Execute[('chmod',
> > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > > > >
> > > > > > 2017-04-25 14:12:53,824
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > Execute['cd
> > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> > > ambari-sudo.sh
> > > > cp
> > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > > > > >
> > > > > > 2017-04-25 14:13:00,439
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > > > > ['delete']}
> > > > > >
> > > > > > 2017-04-25 14:13:00,440
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > > > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
> and
> > > > all
> > > > > > its content
> > > > > >
> > > > > > 2017-04-25 14:13:00,620
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
> 'cd_access':
> > > > 'a'}
> > > > > >
> > > > > > 2017-04-25 14:13:00,621
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Execute[('chmod',
> > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > > > >
> > > > > > 2017-04-25 14:13:00,682
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Initializing
> > > > 2
> > > > > > repositories
> > > > > >
> > > > > > 2017-04-25 14:13:00,683
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > > > ['create'],
> > > > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> endif
> > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > > > > 'mirror_list': None}
> > > > > >
> > > > > > 2017-04-25 14:13:00,709
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content':
> InlineTemplate(...)}
> > > > > >
> > > > > > 2017-04-25 14:13:00,711
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > > > >
> > > > > > 2017-04-25 14:13:00,711
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
> > http://public-repo-1.
> >
> > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> > > > ['create'],
> > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> endif
> > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> > 'HDP-UTILS',
> > > > > > 'mirror_list': None}
> > > > > >
> > > > > > 2017-04-25 14:13:00,717
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > > > InlineTemplate(...)}
> > > > > >
> > > > > > 2017-04-25 14:13:00,717
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > > > > >
> > > > > > 2017-04-25 14:13:00,718
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Package['unzip']
> > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > >
> > > > > > 2017-04-25 14:13:00,846
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Installing
> > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > > > >
> > > > > > 2017-04-25 14:13:03,413
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > Package['curl']
> > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > >
> > > > > > 2017-04-25 14:13:03,429
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > > > > installation of existing package curl
> > > > > >
> > > > > > 2017-04-25 14:13:03,430
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > > > > 'retry_count': 5}
> > > > > >
> > > > > > 2017-04-25 14:13:03,451
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> Installing
> > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
> hdp-select')
> > > > > >
> > > > > > 2017-04-25 14:13:04,948
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install
> ES
> > > > Data
> > > > > > Node
> > > > > >
> > > > > > 2017-04-25 14:13:04,952
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> > > > False,
> > > > > > 'retry_count': 5}
> > > > > >
> > > > > > 2017-04-25 14:13:05,086
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> -
> Installing
> > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > > > > elasticsearch-2.3.3')
> > > > > >
> > > > > >
> > > > > >
> > > > > > Command failed after 1 tries
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
> > ottobackwards@gmail.com)
> >
> > > > > > wrote:
> > > > > >
> > > > > > Yes, I’m going to run through again and get the logs. The basic
> > issue
> > > > > is,
> > > > > > no repo tab -> if I should have one,
> > > > > > and the ES yum command failing with nothing to do, which stops
> > > > deployment
> > > > > > on all the nodes.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > > > michael.miklavcic@gmail.com) wrote:
> > > > > >
> > > > > > I've been doing an internal hwx deployment (HCP) recently for
> > testing
> > > > > > purposes. I have some ES tweaks to work out, but other than that
> it
> > > > has
> > > > > > been running smoothly. Can you share any log info or screenshots?
> > > > > >
> > > > > > Mike
> > > > > >
> > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ottobackwards@gmail.com
> >
> > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > > Should I see the repo tab in the ambari wizard for metron at
> this
> > > > > point?
> > > > > > > Asking for a friend who can’t get the wizard to work :)
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > >
> >
> --
>
> Jon
>
>
Re: Ambari Wizard: Repo Tab
Posted by Michael Miklavcic <mi...@gmail.com>.
A couple more things you should check, given this setup:
- 1 master node (not as data node)
- 2 data nodes
I'd use something like this:
- gateway_recover_after_data_nodes=1 or 2
- index_number_of_replicas=0 or 1 (2 data nodes, with 2 total copies of
the index - if you set this value to 2, it would need 1 more data node to
attain 1 orig index + 2 replicas)
On Wed, Apr 26, 2017 at 12:02 PM, Otto Fowler <ot...@gmail.com>
wrote:
> Actually never mind. I am not sure how to read the exception. Is it
> failing to read the configuration?
>
>
> On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> So the elasticsearch service is disabled. That is the issue. When ES
> calls get status ( service elasticsearch status ) it fails.
> I don’t know systemd too well, but does that mean we didn’t do the equiv
> of chkconfig on?
>
>
>
> On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> Getting the same error in ES as before from what I can see
>
>
> On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> I still can’t keep ES running, it dies.
> Also, I’m getting exceptions from enrichment trying to restart when the
> topology wasn’t running.
>
> Is there some coordination in this work? Should I just log jiras?
> If someone is already sussing out ES, we can sync up
>
>
>
> On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com) wrote:
>
> I'm also interested to know why that's important at such a small scale.
>
> Jon
>
> On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com>
> wrote:
>
> > I am following
> >
> > https://community.hortonworks.com/articles/60805/deploying-
> a-fresh-metron-cluster-using-ambari-serv.html
> > I DO have master and data node together.
> > Why is that a problem?
> >
> > I will try again with master and data node separate.
> >
> >
> > On April 26, 2017 at 10:41:59, Michael Miklavcic (
> > michael.miklavcic@gmail.com) wrote:
> >
> > Hey Otto,
> >
> > How do you have the ES nodes configured? For a base install I would
> setup 1
> > master (NOT as data node) and 2 data nodes (NOT on the same node as the
> > master). This is the install configuration I got working. You can also
> > modify some configuration properties around master node as data node,
> index
> > replicas, and gateway recovery to get it working differently, but this is
> > what will work OOTB with the default config settings from the mpack. If
> > you've already setup a master node and a data node on the same host,
> we'll
> > need to re-install.
> >
> > Mike
> >
> >
> > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > File
> > > "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> > line
> > > 293, in _call
> > >
> > > raise ExecutionFailed(err_msg, code, out, err)
> > >
> > > ExecutionFailed: Execution of 'service elasticsearch status' returned
> 3.
> > ●
> > > elasticsearch.service - Elasticsearch
> > >
> > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> > > disabled;
> > > vendor preset: disabled)
> > >
> > > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58 EDT
> > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
> > >
> > > Docs: http://www.elastic.co
> > >
> > > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch
> > > -Des.pidfile=${PID_DIR}/elasticsearch.pid -Des.default.path.home=${ES_
> > > HOME}
> > > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR}
> > > -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
> > >
> > > Process: 16819
> > > ExecStartPre=/usr/share/elasticsearch/bin/
> elasticsearch-systemd-pre-exec
> > > (code=exited, status=0/SUCCESS)
> > >
> > > Main PID: 16821 (code=exited, status=1/FAILURE)
> > >
> > >
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > > org.elasticsearch.common.settings.Settings$Builder.
> > > loadFromStream(Settings.java:1080)
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > > org.elasticsearch.common.settings.Settings$Builder.
> > > loadFromPath(Settings.java:1067)
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > > org.elasticsearch.node.internal.InternalSettingsPreparer.
> > > prepareEnvironment(InternalSettingsPreparer.java:88)
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > > org.elasticsearch.bootstrap.Bootstrap.initialSettings(
> Bootstrap.java:202)
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > > org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
> Refer
> > > to
> > > the log for complete error details.
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > elasticsearch.service: main process exited, code=exited,
> status=1/FAILURE
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> > > elasticsearch.service entered failed state.
> > >
> > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > > elasticsearch.service failed.
> > >
> > > INFO 2017-04-26 08:55:19,962
> > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
> > >
> > >
> > > On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > Can you describe what ES issues you are working on?
> > > Fixing the repos got everything installed, but my ES components don’t
> > stay
> > > running.
> > > I need to harvest the errors.
> > >
> > >
> > >
> > > On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > Nm. sorry. I fixed it.
> > >
> > >
> > >
> > > On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > Ok, now I see the repos in the ‘pick version’ screen, but it is erring
> on
> > > the f://localrepo
> > > even though the folder exists, there is no repodata/repomd.xml.
> > >
> > > What is the command to create a local repo?
> > >
> > >
> > >
> > > On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > I was going by the HW community page.
> > >
> > > Ok, Let me try it
> > >
> > >
> > >
> > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com)
> wrote:
> > >
> > > That would do it. It requires 2.4.2+. I would have sworn I put that in
> > the
> > > README, but I must have only annotated the PR. :(
> > >
> > > I'll get that in asap.
> > >
> > > -D...
> > >
> > >
> > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
> > > wrote:
> > >
> > > > 2.4.1 as a matter of fact
> > > >
> > > >
> > > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com)
> > wrote:
> > > >
> > > > Any chance they're running Ambari < 2.4.2?
> > > >
> > > > -D...
> > > >
> > > >
> > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > > > michael.miklavcic@gmail.com> wrote:
> > > >
> > > > > Hey Otto, I don't have wizard screenshots in front of me right now
> to
> > > > say
> > > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't run
> > into
> > > > > anything like that, but next time I run through the install I can
> > take
> > > > > another look.
> > > > >
> > > > > https://github.com/apache/incubator-metron/blob/master/
> > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> > > > >
> > > > >
> > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
> > ottobackwards@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > stderr:
> > > > > >
> > > > > > Traceback (most recent call last):
> > > > > >
> > > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > > > > >
> > > > > > Elasticsearch().execute()
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > libraries/script/script.py",
> > > > > > line 280, in execute
> > > > > >
> > > > > > method(env)
> > > > > >
> > > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > > > > >
> > > > > > self.install_packages(env)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > libraries/script/script.py",
> > > > > > line 567, in install_packages
> > > > > >
> > > > > > retry_count=agent_stack_retry_count)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/base.py",
> > > > > > line 155, in __init__
> > > > > >
> > > > > > self.env.run()
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/environment.py",
> > > > > > line 160, in run
> > > > > >
> > > > > > self.run_action(resource, action)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/environment.py",
> > > > > > line 124, in run_action
> > > > > >
> > > > > > provider_action()
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/providers/package/__init__.py", line 54, in action_install
> > > > > >
> > > > > > self.install_package(package_name, self.resource.use_repos,
> > > > > > self.resource.skip_repos)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/providers/package/yumrpm.py", line 49, in install_package
> > > > > >
> > > > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > > > logoutput=self.get_logoutput())
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/providers/package/__init__.py", line 83, in
> > > > > checked_call_with_retries
> > > > > >
> > > > > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > > core/providers/package/__init__.py", line 91, in
> _call_with_retries
> > > > > >
> > > > > > code, out = func(cmd, **kwargs)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/shell.py",
> > > > > > line 71, in inner
> > > > > >
> > > > > > result = function(command, **kwargs)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/shell.py",
> > > > > > line 93, in checked_call
> > > > > >
> > > > > > tries=tries, try_sleep=try_sleep)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/shell.py",
> > > > > > line 141, in _call_wrapper
> > > > > >
> > > > > > result = _call(command, **kwargs_copy)
> > > > > >
> > > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/shell.py",
> > > > > > line 294, in _call
> > > > > >
> > > > > > raise Fail(err_msg)
> > > > > >
> > > > > > resource_management.core.exceptions.Fail: Execution of
> > '/usr/bin/yum
> > > > -d
> > > > > 0
> > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing
> to
> > do
> > > > > >
> > > > > > stdout:
> > > > > >
> > > > > > 2017-04-25 14:12:48,669
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using
> > hadoop
> > > > > conf
> > > > > > dir: /usr/hdp/current/hadoop-client/conf
> > > > > >
> > > > > > 2017-04-25 14:12:48,671
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['metron']
> > > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,673
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > group
> > > > > > Group['metron']
> > > > > >
> > > > > > 2017-04-25 14:12:48,740
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['livy']
> > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,741
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > group
> > > > > > Group['livy']
> > > > > >
> > > > > > 2017-04-25 14:12:48,774
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > > Group['elasticsearch'] {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,775
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > group
> > > > > > Group['elasticsearch']
> > > > > >
> > > > > > 2017-04-25 14:12:48,812
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['spark']
> > > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,812
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > group
> > > > > > Group['spark']
> > > > > >
> > > > > > 2017-04-25 14:12:48,848
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['zeppelin']
> > > > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,849
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > group
> > > > > > Group['zeppelin']
> > > > > >
> > > > > > 2017-04-25 14:12:48,884
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['hadoop']
> > > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,885
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > group
> > > > > > Group['hadoop']
> > > > > >
> > > > > > 2017-04-25 14:12:48,921
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['kibana']
> > > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,922
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > group
> > > > > > Group['kibana']
> > > > > >
> > > > > > 2017-04-25 14:12:48,959
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['users']
> > > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:48,960
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > User['hive']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:48,961
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> > user
> > > > > > User['hive']
> > > > > >
> > > > > > 2017-04-25 14:12:49,227
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['storm']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,227
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['storm']
> > > > > >
> > > > > > 2017-04-25 14:12:49,388
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['zookeeper']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,388
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['zookeeper']
> > > > > >
> > > > > > 2017-04-25 14:12:49,518
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> True,
> > > > > > 'groups': [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,518
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['infra-solr']
> > > > > >
> > > > > > 2017-04-25 14:12:49,578
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['ams']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,578
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['ams']
> > > > > >
> > > > > > 2017-04-25 14:12:49,625
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['tez']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'users']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,626
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['tez']
> > > > > >
> > > > > > 2017-04-25 14:12:49,660
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['zeppelin']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,660
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['zeppelin']
> > > > > >
> > > > > > 2017-04-25 14:12:49,705
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['metron']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,705
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['metron']
> > > > > >
> > > > > > 2017-04-25 14:12:49,760
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['livy']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,760
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['livy']
> > > > > >
> > > > > > 2017-04-25 14:12:49,803
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> > > True,
> > > > > > 'groups': [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,804
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['elasticsearch']
> > > > > >
> > > > > > 2017-04-25 14:12:49,837
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['spark']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,837
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['spark']
> > > > > >
> > > > > > 2017-04-25 14:12:49,885
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['ambari-qa']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'users']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,885
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['ambari-qa']
> > > > > >
> > > > > > 2017-04-25 14:12:49,928
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['flume']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,928
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['flume']
> > > > > >
> > > > > > 2017-04-25 14:12:49,972
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['kafka']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:49,973
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> > user
> > > > > > User['kafka']
> > > > > >
> > > > > > 2017-04-25 14:12:50,025
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['hdfs']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:50,026
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > user
> > > > > > User['hdfs']
> > > > > >
> > > > > > 2017-04-25 14:12:50,075
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['yarn']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:50,075
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > user
> > > > > > User['yarn']
> > > > > >
> > > > > > 2017-04-25 14:12:50,125
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['kibana']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:50,126
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > user
> > > > > > User['kibana']
> > > > > >
> > > > > > 2017-04-25 14:12:50,175
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > User['mapred']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:50,176
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > user
> > > > > > User['mapred']
> > > > > >
> > > > > > 2017-04-25 14:12:50,223
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['hbase']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:50,224
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > user
> > > > > > User['hbase']
> > > > > >
> > > > > > 2017-04-25 14:12:50,271
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['hcat']
> > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > > [u'hadoop']}
> > > > > >
> > > > > > 2017-04-25 14:12:50,272
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > user
> > > > > > User['hcat']
> > > > > >
> > > > > > 2017-04-25 14:12:50,320
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > >
> > > > > > 2017-04-25 14:12:50,333
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it
> doesn't
> > > > exist
> > > > > >
> > > > > > 2017-04-25 14:12:50,334
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644
> to
> > > 555
> > > > > >
> > > > > > 2017-04-25 14:12:50,335
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if':
> '(test
> > > > $(id
> > > > > -u
> > > > > > ambari-qa) -gt 1000) || (false)'}
> > > > > >
> > > > > > 2017-04-25 14:12:50,364
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > > > > >
> > > > > > 2017-04-25 14:12:50,365
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
> 'create_parents':
> > > > True,
> > > > > > 'mode': 0775, 'cd_access': 'a'}
> > > > > >
> > > > > > 2017-04-25 14:12:50,366
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > > > > >
> > > > > > 2017-04-25 14:12:50,366
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > owner
> > > > > > for /tmp/hbase-hbase from 0 to hbase
> > > > > >
> > > > > > 2017-04-25 14:12:50,366
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > > permission for /tmp/hbase-hbase from 755 to 775
> > > > > >
> > > > > > 2017-04-25 14:12:50,368
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > > >
> > > > > > 2017-04-25 14:12:50,369
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > >
> > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> hbase-hbase']
> > > >
> > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > > > >
> > > > > > 2017-04-25 14:12:50,393
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > > >
> > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/
> hbase-hbase']
> > > >
> > > > > > due to not_if
> > > > > >
> > > > > > 2017-04-25 14:12:50,394
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Group['hdfs']
> > > > {}
> > > > > >
> > > > > > 2017-04-25 14:12:50,395
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> > group
> > > > > > Group['hdfs']
> > > > > >
> > > > > > 2017-04-25 14:12:50,430
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['hdfs']
> > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > > > > >
> > > > > > 2017-04-25 14:12:50,431
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Modifying
> > > > user
> > > > > > hdfs
> > > > > >
> > > > > > 2017-04-25 14:12:50,474
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > > > > >
> > > > > > 2017-04-25 14:12:50,474
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > > > >
> > > > > > 2017-04-25 14:12:50,474
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > > > >
> > > > > > 2017-04-25 14:12:50,475
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > > > {'owner':
> > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > > > >
> > > > > > 2017-04-25 14:12:50,476
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
> > > tmpdir']
> > > >
> > > > > > since it doesn't exist.
> > > > > >
> > > > > > 2017-04-25 14:12:50,476
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > owner
> > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
> hdfs
> > > > > >
> > > > > > 2017-04-25 14:12:50,476
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > group
> > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
> > hadoop
> > > > > >
> > > > > > 2017-04-25 14:12:50,477
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
> from
> > > > 755
> > > > > > to 1777
> > > > > >
> > > > > > 2017-04-25 14:12:50,477
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > > > {'create_parents': True}
> > > > > >
> > > > > > 2017-04-25 14:12:50,478
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > > directory Directory['/var/lib/ambari-
> agent/tmp/AMBARI-artifacts/']
> > > > since
> > > > > > it doesn't exist.
> > > > > >
> > > > > > 2017-04-25 14:12:50,479
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > > > {'content':
> > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > > > >
> > > > > > 2017-04-25 14:12:50,504
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Downloading
> > > > the
> > > > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > > > linux-x64.tar.gz
> > > > > >
> > > > > > 2017-04-25 14:12:53,793
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > > Directory['/usr/jdk64'] {}
> > > > > >
> > > > > > 2017-04-25 14:12:53,793
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > > > >
> > > > > > 2017-04-25 14:12:53,794
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > Execute[('chmod',
> > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > > > >
> > > > > > 2017-04-25 14:12:53,824
> > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > Execute['cd
> > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> > > ambari-sudo.sh
> > > > cp
> > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > > > > >
> > > > > > 2017-04-25 14:13:00,439
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > > > > ['delete']}
> > > > > >
> > > > > > 2017-04-25 14:13:00,440
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > > > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
> and
> > > > all
> > > > > > its content
> > > > > >
> > > > > > 2017-04-25 14:13:00,620
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
> 'cd_access':
> > > > 'a'}
> > > > > >
> > > > > > 2017-04-25 14:13:00,621
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Execute[('chmod',
> > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > > > >
> > > > > > 2017-04-25 14:13:00,682
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Initializing
> > > > 2
> > > > > > repositories
> > > > > >
> > > > > > 2017-04-25 14:13:00,683
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > > > ['create'],
> > > > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> endif
> > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > > > > 'mirror_list': None}
> > > > > >
> > > > > > 2017-04-25 14:13:00,709
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content':
> InlineTemplate(...)}
> > > > > >
> > > > > > 2017-04-25 14:13:00,711
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > > > >
> > > > > > 2017-04-25 14:13:00,711
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
> > http://public-repo-1.
> >
> > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> > > > ['create'],
> > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
> endif
> > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> > 'HDP-UTILS',
> > > > > > 'mirror_list': None}
> > > > > >
> > > > > > 2017-04-25 14:13:00,717
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > > > InlineTemplate(...)}
> > > > > >
> > > > > > 2017-04-25 14:13:00,717
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > > > > >
> > > > > > 2017-04-25 14:13:00,718
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Package['unzip']
> > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > >
> > > > > > 2017-04-25 14:13:00,846
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Installing
> > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > > > >
> > > > > > 2017-04-25 14:13:03,413
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > Package['curl']
> > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > > >
> > > > > > 2017-04-25 14:13:03,429
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > > > > installation of existing package curl
> > > > > >
> > > > > > 2017-04-25 14:13:03,430
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > > > > 'retry_count': 5}
> > > > > >
> > > > > > 2017-04-25 14:13:03,451
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> Installing
> > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
> hdp-select')
> > > > > >
> > > > > > 2017-04-25 14:13:04,948
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install
> ES
> > > > Data
> > > > > > Node
> > > > > >
> > > > > > 2017-04-25 14:13:04,952
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> > > > False,
> > > > > > 'retry_count': 5}
> > > > > >
> > > > > > 2017-04-25 14:13:05,086
> > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> -
> Installing
> > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > > > > elasticsearch-2.3.3')
> > > > > >
> > > > > >
> > > > > >
> > > > > > Command failed after 1 tries
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
> > ottobackwards@gmail.com)
> >
> > > > > > wrote:
> > > > > >
> > > > > > Yes, I’m going to run through again and get the logs. The basic
> > issue
> > > > > is,
> > > > > > no repo tab -> if I should have one,
> > > > > > and the ES yum command failing with nothing to do, which stops
> > > > deployment
> > > > > > on all the nodes.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > > > michael.miklavcic@gmail.com) wrote:
> > > > > >
> > > > > > I've been doing an internal hwx deployment (HCP) recently for
> > testing
> > > > > > purposes. I have some ES tweaks to work out, but other than that
> it
> > > > has
> > > > > > been running smoothly. Can you share any log info or screenshots?
> > > > > >
> > > > > > Mike
> > > > > >
> > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ottobackwards@gmail.com
> >
> > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > > Should I see the repo tab in the ambari wizard for metron at
> this
> > > > > point?
> > > > > > > Asking for a friend who can’t get the wizard to work :)
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > >
> >
> --
>
> Jon
>
>
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
Actually never mind. I am not sure how to read the exception. Is it
failing to read the configuration?
On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwards@gmail.com) wrote:
So the elasticsearch service is disabled. That is the issue. When ES
calls get status ( service elasticsearch status ) it fails.
I don’t know systemd too well, but does that mean we didn’t do the equiv of
chkconfig on?
On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com) wrote:
Getting the same error in ES as before from what I can see
On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com) wrote:
I still can’t keep ES running, it dies.
Also, I’m getting exceptions from enrichment trying to restart when the
topology wasn’t running.
Is there some coordination in this work? Should I just log jiras?
If someone is already sussing out ES, we can sync up
On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com) wrote:
I'm also interested to know why that's important at such a small scale.
Jon
On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com> wrote:
> I am following
>
>
https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html
> I DO have master and data node together.
> Why is that a problem?
>
> I will try again with master and data node separate.
>
>
> On April 26, 2017 at 10:41:59, Michael Miklavcic (
> michael.miklavcic@gmail.com) wrote:
>
> Hey Otto,
>
> How do you have the ES nodes configured? For a base install I would setup
1
> master (NOT as data node) and 2 data nodes (NOT on the same node as the
> master). This is the install configuration I got working. You can also
> modify some configuration properties around master node as data node,
index
> replicas, and gateway recovery to get it working differently, but this is
> what will work OOTB with the default config settings from the mpack. If
> you've already setup a master node and a data node on the same host, we'll
> need to re-install.
>
> Mike
>
>
> On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > File
> > "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line
> > 293, in _call
> >
> > raise ExecutionFailed(err_msg, code, out, err)
> >
> > ExecutionFailed: Execution of 'service elasticsearch status' returned 3.
> ●
> > elasticsearch.service - Elasticsearch
> >
> > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> > disabled;
> > vendor preset: disabled)
> >
> > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58 EDT
> > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
> >
> > Docs: http://www.elastic.co
> >
> > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch
> > -Des.pidfile=${PID_DIR}/elasticsearch.pid -Des.default.path.home=${ES_
> > HOME}
> > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR}
> > -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
> >
> > Process: 16819
> > ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec
> > (code=exited, status=0/SUCCESS)
> >
> > Main PID: 16821 (code=exited, status=1/FAILURE)
> >
> >
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.common.settings.Settings$Builder.
> > loadFromStream(Settings.java:1080)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.common.settings.Settings$Builder.
> > loadFromPath(Settings.java:1067)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.node.internal.InternalSettingsPreparer.
> > prepareEnvironment(InternalSettingsPreparer.java:88)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> >
org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: Refer
> > to
> > the log for complete error details.
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > elasticsearch.service: main process exited, code=exited,
status=1/FAILURE
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> > elasticsearch.service entered failed state.
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > elasticsearch.service failed.
> >
> > INFO 2017-04-26 08:55:19,962
> > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
> >
> >
> > On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Can you describe what ES issues you are working on?
> > Fixing the repos got everything installed, but my ES components don’t
> stay
> > running.
> > I need to harvest the errors.
> >
> >
> >
> > On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Nm. sorry. I fixed it.
> >
> >
> >
> > On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Ok, now I see the repos in the ‘pick version’ screen, but it is erring
on
> > the f://localrepo
> > even though the folder exists, there is no repodata/repomd.xml.
> >
> > What is the command to create a local repo?
> >
> >
> >
> > On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > I was going by the HW community page.
> >
> > Ok, Let me try it
> >
> >
> >
> > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com) wrote:
> >
> > That would do it. It requires 2.4.2+. I would have sworn I put that in
> the
> > README, but I must have only annotated the PR. :(
> >
> > I'll get that in asap.
> >
> > -D...
> >
> >
> > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > 2.4.1 as a matter of fact
> > >
> > >
> > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com)
> wrote:
> > >
> > > Any chance they're running Ambari < 2.4.2?
> > >
> > > -D...
> > >
> > >
> > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > > michael.miklavcic@gmail.com> wrote:
> > >
> > > > Hey Otto, I don't have wizard screenshots in front of me right now
to
> > > say
> > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't run
> into
> > > > anything like that, but next time I run through the install I can
> take
> > > > another look.
> > > >
> > > > https://github.com/apache/incubator-metron/blob/master/
> > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> > > >
> > > >
> > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
> ottobackwards@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > stderr:
> > > > >
> > > > > Traceback (most recent call last):
> > > > >
> > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > > > >
> > > > > Elasticsearch().execute()
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > libraries/script/script.py",
> > > > > line 280, in execute
> > > > >
> > > > > method(env)
> > > > >
> > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > > > >
> > > > > self.install_packages(env)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > libraries/script/script.py",
> > > > > line 567, in install_packages
> > > > >
> > > > > retry_count=agent_stack_retry_count)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/base.py",
> > > > > line 155, in __init__
> > > > >
> > > > > self.env.run()
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/environment.py",
> > > > > line 160, in run
> > > > >
> > > > > self.run_action(resource, action)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/environment.py",
> > > > > line 124, in run_action
> > > > >
> > > > > provider_action()
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/__init__.py", line 54, in action_install
> > > > >
> > > > > self.install_package(package_name, self.resource.use_repos,
> > > > > self.resource.skip_repos)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/yumrpm.py", line 49, in install_package
> > > > >
> > > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > > logoutput=self.get_logoutput())
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/__init__.py", line 83, in
> > > > checked_call_with_retries
> > > > >
> > > > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/__init__.py", line 91, in
_call_with_retries
> > > > >
> > > > > code, out = func(cmd, **kwargs)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 71, in inner
> > > > >
> > > > > result = function(command, **kwargs)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 93, in checked_call
> > > > >
> > > > > tries=tries, try_sleep=try_sleep)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 141, in _call_wrapper
> > > > >
> > > > > result = _call(command, **kwargs_copy)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 294, in _call
> > > > >
> > > > > raise Fail(err_msg)
> > > > >
> > > > > resource_management.core.exceptions.Fail: Execution of
> '/usr/bin/yum
> > > -d
> > > > 0
> > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to
> do
> > > > >
> > > > > stdout:
> > > > >
> > > > > 2017-04-25 14:12:48,669
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using
> hadoop
> > > > conf
> > > > > dir: /usr/hdp/current/hadoop-client/conf
> > > > >
> > > > > 2017-04-25 14:12:48,671
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['metron']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,673
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['metron']
> > > > >
> > > > > 2017-04-25 14:12:48,740
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['livy']
> > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,741
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['livy']
> > > > >
> > > > > 2017-04-25 14:12:48,774
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['elasticsearch'] {}
> > > > >
> > > > > 2017-04-25 14:12:48,775
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['elasticsearch']
> > > > >
> > > > > 2017-04-25 14:12:48,812
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['spark']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,812
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['spark']
> > > > >
> > > > > 2017-04-25 14:12:48,848
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['zeppelin']
> > > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,849
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['zeppelin']
> > > > >
> > > > > 2017-04-25 14:12:48,884
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['hadoop']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,885
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['hadoop']
> > > > >
> > > > > 2017-04-25 14:12:48,921
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['kibana']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,922
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['kibana']
> > > > >
> > > > > 2017-04-25 14:12:48,959
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['users']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,960
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> User['hive']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:48,961
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> user
> > > > > User['hive']
> > > > >
> > > > > 2017-04-25 14:12:49,227
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['storm']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,227
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['storm']
> > > > >
> > > > > 2017-04-25 14:12:49,388
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['zookeeper']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,388
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['zookeeper']
> > > > >
> > > > > 2017-04-25 14:12:49,518
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
True,
> > > > > 'groups': [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,518
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['infra-solr']
> > > > >
> > > > > 2017-04-25 14:12:49,578
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['ams']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,578
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['ams']
> > > > >
> > > > > 2017-04-25 14:12:49,625
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['tez']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'users']}
> > > > >
> > > > > 2017-04-25 14:12:49,626
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['tez']
> > > > >
> > > > > 2017-04-25 14:12:49,660
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['zeppelin']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,660
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['zeppelin']
> > > > >
> > > > > 2017-04-25 14:12:49,705
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['metron']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,705
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['metron']
> > > > >
> > > > > 2017-04-25 14:12:49,760
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['livy']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,760
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['livy']
> > > > >
> > > > > 2017-04-25 14:12:49,803
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> > True,
> > > > > 'groups': [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,804
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['elasticsearch']
> > > > >
> > > > > 2017-04-25 14:12:49,837
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['spark']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,837
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['spark']
> > > > >
> > > > > 2017-04-25 14:12:49,885
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['ambari-qa']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'users']}
> > > > >
> > > > > 2017-04-25 14:12:49,885
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['ambari-qa']
> > > > >
> > > > > 2017-04-25 14:12:49,928
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['flume']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,928
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['flume']
> > > > >
> > > > > 2017-04-25 14:12:49,972
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['kafka']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,973
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['kafka']
> > > > >
> > > > > 2017-04-25 14:12:50,025
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hdfs']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,026
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['hdfs']
> > > > >
> > > > > 2017-04-25 14:12:50,075
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['yarn']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,075
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['yarn']
> > > > >
> > > > > 2017-04-25 14:12:50,125
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['kibana']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,126
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['kibana']
> > > > >
> > > > > 2017-04-25 14:12:50,175
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['mapred']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,176
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['mapred']
> > > > >
> > > > > 2017-04-25 14:12:50,223
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['hbase']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,224
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['hbase']
> > > > >
> > > > > 2017-04-25 14:12:50,271
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hcat']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,272
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['hcat']
> > > > >
> > > > > 2017-04-25 14:12:50,320
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > >
> > > > > 2017-04-25 14:12:50,333
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
> > > exist
> > > > >
> > > > > 2017-04-25 14:12:50,334
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to
> > 555
> > > > >
> > > > > 2017-04-25 14:12:50,335
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
> > > $(id
> > > > -u
> > > > > ambari-qa) -gt 1000) || (false)'}
> > > > >
> > > > > 2017-04-25 14:12:50,364
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > > > >
> > > > > 2017-04-25 14:12:50,365
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
> > > True,
> > > > > 'mode': 0775, 'cd_access': 'a'}
> > > > >
> > > > > 2017-04-25 14:12:50,366
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,366
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > owner
> > > > > for /tmp/hbase-hbase from 0 to hbase
> > > > >
> > > > > 2017-04-25 14:12:50,366
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > permission for /tmp/hbase-hbase from 755 to 775
> > > > >
> > > > > 2017-04-25 14:12:50,368
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > >
> > > > > 2017-04-25 14:12:50,369
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > >
> > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> > >
> > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > > >
> > > > > 2017-04-25 14:12:50,393
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > >
> > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> > >
> > > > > due to not_if
> > > > >
> > > > > 2017-04-25 14:12:50,394
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Group['hdfs']
> > > {}
> > > > >
> > > > > 2017-04-25 14:12:50,395
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> group
> > > > > Group['hdfs']
> > > > >
> > > > > 2017-04-25 14:12:50,430
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hdfs']
> > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > > > >
> > > > > 2017-04-25 14:12:50,431
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
> > > user
> > > > > hdfs
> > > > >
> > > > > 2017-04-25 14:12:50,474
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > > > >
> > > > > 2017-04-25 14:12:50,474
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > > >
> > > > > 2017-04-25 14:12:50,474
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,475
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > > {'owner':
> > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > > >
> > > > > 2017-04-25 14:12:50,476
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
> > tmpdir']
> > >
> > > > > since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,476
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > owner
> > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
> > > > >
> > > > > 2017-04-25 14:12:50,476
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > group
> > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
> hadoop
> > > > >
> > > > > 2017-04-25 14:12:50,477
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
from
> > > 755
> > > > > to 1777
> > > > >
> > > > > 2017-04-25 14:12:50,477
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > > {'create_parents': True}
> > > > >
> > > > > 2017-04-25 14:12:50,478
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > since
> > > > > it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,479
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > > {'content':
> > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > > >
> > > > > 2017-04-25 14:12:50,504
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Downloading
> > > the
> > > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > > linux-x64.tar.gz
> > > > >
> > > > > 2017-04-25 14:12:53,793
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > Directory['/usr/jdk64'] {}
> > > > >
> > > > > 2017-04-25 14:12:53,793
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:53,794
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > Execute[('chmod',
> > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > > >
> > > > > 2017-04-25 14:12:53,824
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> Execute['cd
> > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> > ambari-sudo.sh
> > > cp
> > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > > > >
> > > > > 2017-04-25 14:13:00,439
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > > > ['delete']}
> > > > >
> > > > > 2017-04-25 14:13:00,440
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
and
> > > all
> > > > > its content
> > > > >
> > > > > 2017-04-25 14:13:00,620
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
'cd_access':
> > > 'a'}
> > > > >
> > > > > 2017-04-25 14:13:00,621
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Execute[('chmod',
> > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > > >
> > > > > 2017-04-25 14:13:00,682
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Initializing
> > > 2
> > > > > repositories
> > > > >
> > > > > 2017-04-25 14:13:00,683
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > > ['create'],
> > > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > > > 'mirror_list': None}
> > > > >
> > > > > 2017-04-25 14:13:00,709
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
> > > > >
> > > > > 2017-04-25 14:13:00,711
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > > >
> > > > > 2017-04-25 14:13:00,711
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
> http://public-repo-1.
>
> > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> > > ['create'],
> > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> 'HDP-UTILS',
> > > > > 'mirror_list': None}
> > > > >
> > > > > 2017-04-25 14:13:00,717
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > > InlineTemplate(...)}
> > > > >
> > > > > 2017-04-25 14:13:00,717
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > > > >
> > > > > 2017-04-25 14:13:00,718
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Package['unzip']
> > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:00,846
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > > >
> > > > > 2017-04-25 14:13:03,413
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Package['curl']
> > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:03,429
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > > > installation of existing package curl
> > > > >
> > > > > 2017-04-25 14:13:03,430
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > > > 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:03,451
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
hdp-select')
> > > > >
> > > > > 2017-04-25 14:13:04,948
> > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES
> > > Data
> > > > > Node
> > > > >
> > > > > 2017-04-25 14:13:04,952
> > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> > > False,
> > > > > 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:05,086
> > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > > > elasticsearch-2.3.3')
> > > > >
> > > > >
> > > > >
> > > > > Command failed after 1 tries
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
> ottobackwards@gmail.com)
>
> > > > > wrote:
> > > > >
> > > > > Yes, I’m going to run through again and get the logs. The basic
> issue
> > > > is,
> > > > > no repo tab -> if I should have one,
> > > > > and the ES yum command failing with nothing to do, which stops
> > > deployment
> > > > > on all the nodes.
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > > michael.miklavcic@gmail.com) wrote:
> > > > >
> > > > > I've been doing an internal hwx deployment (HCP) recently for
> testing
> > > > > purposes. I have some ES tweaks to work out, but other than that
it
> > > has
> > > > > been running smoothly. Can you share any log info or screenshots?
> > > > >
> > > > > Mike
> > > > >
> > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Hi,
> > > > > > Should I see the repo tab in the ambari wizard for metron at
this
> > > > point?
> > > > > > Asking for a friend who can’t get the wizard to work :)
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> >
>
--
Jon
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
So the elasticsearch service is disabled. That is the issue. When ES
calls get status ( service elasticsearch status ) it fails.
I don’t know systemd too well, but does that mean we didn’t do the equiv of
chkconfig on?
On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwards@gmail.com) wrote:
Getting the same error in ES as before from what I can see
On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com) wrote:
I still can’t keep ES running, it dies.
Also, I’m getting exceptions from enrichment trying to restart when the
topology wasn’t running.
Is there some coordination in this work? Should I just log jiras?
If someone is already sussing out ES, we can sync up
On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com) wrote:
I'm also interested to know why that's important at such a small scale.
Jon
On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com> wrote:
> I am following
>
>
https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html
> I DO have master and data node together.
> Why is that a problem?
>
> I will try again with master and data node separate.
>
>
> On April 26, 2017 at 10:41:59, Michael Miklavcic (
> michael.miklavcic@gmail.com) wrote:
>
> Hey Otto,
>
> How do you have the ES nodes configured? For a base install I would setup
1
> master (NOT as data node) and 2 data nodes (NOT on the same node as the
> master). This is the install configuration I got working. You can also
> modify some configuration properties around master node as data node,
index
> replicas, and gateway recovery to get it working differently, but this is
> what will work OOTB with the default config settings from the mpack. If
> you've already setup a master node and a data node on the same host, we'll
> need to re-install.
>
> Mike
>
>
> On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > File
> > "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line
> > 293, in _call
> >
> > raise ExecutionFailed(err_msg, code, out, err)
> >
> > ExecutionFailed: Execution of 'service elasticsearch status' returned 3.
> ●
> > elasticsearch.service - Elasticsearch
> >
> > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> > disabled;
> > vendor preset: disabled)
> >
> > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58 EDT
> > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
> >
> > Docs: http://www.elastic.co
> >
> > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch
> > -Des.pidfile=${PID_DIR}/elasticsearch.pid -Des.default.path.home=${ES_
> > HOME}
> > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR}
> > -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
> >
> > Process: 16819
> > ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec
> > (code=exited, status=0/SUCCESS)
> >
> > Main PID: 16821 (code=exited, status=1/FAILURE)
> >
> >
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.common.settings.Settings$Builder.
> > loadFromStream(Settings.java:1080)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.common.settings.Settings$Builder.
> > loadFromPath(Settings.java:1067)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.node.internal.InternalSettingsPreparer.
> > prepareEnvironment(InternalSettingsPreparer.java:88)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> >
org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: Refer
> > to
> > the log for complete error details.
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > elasticsearch.service: main process exited, code=exited,
status=1/FAILURE
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> > elasticsearch.service entered failed state.
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > elasticsearch.service failed.
> >
> > INFO 2017-04-26 08:55:19,962
> > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
> >
> >
> > On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Can you describe what ES issues you are working on?
> > Fixing the repos got everything installed, but my ES components don’t
> stay
> > running.
> > I need to harvest the errors.
> >
> >
> >
> > On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Nm. sorry. I fixed it.
> >
> >
> >
> > On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Ok, now I see the repos in the ‘pick version’ screen, but it is erring
on
> > the f://localrepo
> > even though the folder exists, there is no repodata/repomd.xml.
> >
> > What is the command to create a local repo?
> >
> >
> >
> > On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > I was going by the HW community page.
> >
> > Ok, Let me try it
> >
> >
> >
> > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com) wrote:
> >
> > That would do it. It requires 2.4.2+. I would have sworn I put that in
> the
> > README, but I must have only annotated the PR. :(
> >
> > I'll get that in asap.
> >
> > -D...
> >
> >
> > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > 2.4.1 as a matter of fact
> > >
> > >
> > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com)
> wrote:
> > >
> > > Any chance they're running Ambari < 2.4.2?
> > >
> > > -D...
> > >
> > >
> > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > > michael.miklavcic@gmail.com> wrote:
> > >
> > > > Hey Otto, I don't have wizard screenshots in front of me right now
to
> > > say
> > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't run
> into
> > > > anything like that, but next time I run through the install I can
> take
> > > > another look.
> > > >
> > > > https://github.com/apache/incubator-metron/blob/master/
> > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> > > >
> > > >
> > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
> ottobackwards@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > stderr:
> > > > >
> > > > > Traceback (most recent call last):
> > > > >
> > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > > > >
> > > > > Elasticsearch().execute()
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > libraries/script/script.py",
> > > > > line 280, in execute
> > > > >
> > > > > method(env)
> > > > >
> > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > > > >
> > > > > self.install_packages(env)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > libraries/script/script.py",
> > > > > line 567, in install_packages
> > > > >
> > > > > retry_count=agent_stack_retry_count)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/base.py",
> > > > > line 155, in __init__
> > > > >
> > > > > self.env.run()
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/environment.py",
> > > > > line 160, in run
> > > > >
> > > > > self.run_action(resource, action)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/environment.py",
> > > > > line 124, in run_action
> > > > >
> > > > > provider_action()
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/__init__.py", line 54, in action_install
> > > > >
> > > > > self.install_package(package_name, self.resource.use_repos,
> > > > > self.resource.skip_repos)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/yumrpm.py", line 49, in install_package
> > > > >
> > > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > > logoutput=self.get_logoutput())
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/__init__.py", line 83, in
> > > > checked_call_with_retries
> > > > >
> > > > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/__init__.py", line 91, in
_call_with_retries
> > > > >
> > > > > code, out = func(cmd, **kwargs)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 71, in inner
> > > > >
> > > > > result = function(command, **kwargs)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 93, in checked_call
> > > > >
> > > > > tries=tries, try_sleep=try_sleep)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 141, in _call_wrapper
> > > > >
> > > > > result = _call(command, **kwargs_copy)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 294, in _call
> > > > >
> > > > > raise Fail(err_msg)
> > > > >
> > > > > resource_management.core.exceptions.Fail: Execution of
> '/usr/bin/yum
> > > -d
> > > > 0
> > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to
> do
> > > > >
> > > > > stdout:
> > > > >
> > > > > 2017-04-25 14:12:48,669
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using
> hadoop
> > > > conf
> > > > > dir: /usr/hdp/current/hadoop-client/conf
> > > > >
> > > > > 2017-04-25 14:12:48,671
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['metron']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,673
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['metron']
> > > > >
> > > > > 2017-04-25 14:12:48,740
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['livy']
> > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,741
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['livy']
> > > > >
> > > > > 2017-04-25 14:12:48,774
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['elasticsearch'] {}
> > > > >
> > > > > 2017-04-25 14:12:48,775
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['elasticsearch']
> > > > >
> > > > > 2017-04-25 14:12:48,812
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['spark']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,812
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['spark']
> > > > >
> > > > > 2017-04-25 14:12:48,848
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['zeppelin']
> > > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,849
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['zeppelin']
> > > > >
> > > > > 2017-04-25 14:12:48,884
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['hadoop']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,885
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['hadoop']
> > > > >
> > > > > 2017-04-25 14:12:48,921
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['kibana']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,922
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['kibana']
> > > > >
> > > > > 2017-04-25 14:12:48,959
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['users']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,960
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> User['hive']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:48,961
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> user
> > > > > User['hive']
> > > > >
> > > > > 2017-04-25 14:12:49,227
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['storm']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,227
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['storm']
> > > > >
> > > > > 2017-04-25 14:12:49,388
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['zookeeper']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,388
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['zookeeper']
> > > > >
> > > > > 2017-04-25 14:12:49,518
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
True,
> > > > > 'groups': [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,518
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['infra-solr']
> > > > >
> > > > > 2017-04-25 14:12:49,578
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['ams']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,578
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['ams']
> > > > >
> > > > > 2017-04-25 14:12:49,625
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['tez']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'users']}
> > > > >
> > > > > 2017-04-25 14:12:49,626
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['tez']
> > > > >
> > > > > 2017-04-25 14:12:49,660
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['zeppelin']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,660
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['zeppelin']
> > > > >
> > > > > 2017-04-25 14:12:49,705
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['metron']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,705
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['metron']
> > > > >
> > > > > 2017-04-25 14:12:49,760
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['livy']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,760
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['livy']
> > > > >
> > > > > 2017-04-25 14:12:49,803
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> > True,
> > > > > 'groups': [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,804
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['elasticsearch']
> > > > >
> > > > > 2017-04-25 14:12:49,837
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['spark']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,837
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['spark']
> > > > >
> > > > > 2017-04-25 14:12:49,885
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['ambari-qa']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'users']}
> > > > >
> > > > > 2017-04-25 14:12:49,885
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['ambari-qa']
> > > > >
> > > > > 2017-04-25 14:12:49,928
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['flume']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,928
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['flume']
> > > > >
> > > > > 2017-04-25 14:12:49,972
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['kafka']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,973
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['kafka']
> > > > >
> > > > > 2017-04-25 14:12:50,025
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hdfs']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,026
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['hdfs']
> > > > >
> > > > > 2017-04-25 14:12:50,075
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['yarn']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,075
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['yarn']
> > > > >
> > > > > 2017-04-25 14:12:50,125
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['kibana']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,126
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['kibana']
> > > > >
> > > > > 2017-04-25 14:12:50,175
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['mapred']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,176
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['mapred']
> > > > >
> > > > > 2017-04-25 14:12:50,223
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['hbase']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,224
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['hbase']
> > > > >
> > > > > 2017-04-25 14:12:50,271
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hcat']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,272
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['hcat']
> > > > >
> > > > > 2017-04-25 14:12:50,320
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > >
> > > > > 2017-04-25 14:12:50,333
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
> > > exist
> > > > >
> > > > > 2017-04-25 14:12:50,334
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to
> > 555
> > > > >
> > > > > 2017-04-25 14:12:50,335
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
> > > $(id
> > > > -u
> > > > > ambari-qa) -gt 1000) || (false)'}
> > > > >
> > > > > 2017-04-25 14:12:50,364
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > > > >
> > > > > 2017-04-25 14:12:50,365
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
> > > True,
> > > > > 'mode': 0775, 'cd_access': 'a'}
> > > > >
> > > > > 2017-04-25 14:12:50,366
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,366
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > owner
> > > > > for /tmp/hbase-hbase from 0 to hbase
> > > > >
> > > > > 2017-04-25 14:12:50,366
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > permission for /tmp/hbase-hbase from 755 to 775
> > > > >
> > > > > 2017-04-25 14:12:50,368
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > >
> > > > > 2017-04-25 14:12:50,369
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > >
> > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> > >
> > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > > >
> > > > > 2017-04-25 14:12:50,393
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > >
> > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> > >
> > > > > due to not_if
> > > > >
> > > > > 2017-04-25 14:12:50,394
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Group['hdfs']
> > > {}
> > > > >
> > > > > 2017-04-25 14:12:50,395
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> group
> > > > > Group['hdfs']
> > > > >
> > > > > 2017-04-25 14:12:50,430
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hdfs']
> > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > > > >
> > > > > 2017-04-25 14:12:50,431
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
> > > user
> > > > > hdfs
> > > > >
> > > > > 2017-04-25 14:12:50,474
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > > > >
> > > > > 2017-04-25 14:12:50,474
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > > >
> > > > > 2017-04-25 14:12:50,474
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,475
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > > {'owner':
> > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > > >
> > > > > 2017-04-25 14:12:50,476
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
> > tmpdir']
> > >
> > > > > since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,476
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > owner
> > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
> > > > >
> > > > > 2017-04-25 14:12:50,476
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > group
> > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
> hadoop
> > > > >
> > > > > 2017-04-25 14:12:50,477
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
from
> > > 755
> > > > > to 1777
> > > > >
> > > > > 2017-04-25 14:12:50,477
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > > {'create_parents': True}
> > > > >
> > > > > 2017-04-25 14:12:50,478
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > since
> > > > > it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,479
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > > {'content':
> > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > > >
> > > > > 2017-04-25 14:12:50,504
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Downloading
> > > the
> > > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > > linux-x64.tar.gz
> > > > >
> > > > > 2017-04-25 14:12:53,793
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > Directory['/usr/jdk64'] {}
> > > > >
> > > > > 2017-04-25 14:12:53,793
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:53,794
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > Execute[('chmod',
> > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > > >
> > > > > 2017-04-25 14:12:53,824
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> Execute['cd
> > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> > ambari-sudo.sh
> > > cp
> > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > > > >
> > > > > 2017-04-25 14:13:00,439
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > > > ['delete']}
> > > > >
> > > > > 2017-04-25 14:13:00,440
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
and
> > > all
> > > > > its content
> > > > >
> > > > > 2017-04-25 14:13:00,620
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
'cd_access':
> > > 'a'}
> > > > >
> > > > > 2017-04-25 14:13:00,621
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Execute[('chmod',
> > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > > >
> > > > > 2017-04-25 14:13:00,682
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Initializing
> > > 2
> > > > > repositories
> > > > >
> > > > > 2017-04-25 14:13:00,683
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > > ['create'],
> > > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > > > 'mirror_list': None}
> > > > >
> > > > > 2017-04-25 14:13:00,709
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
> > > > >
> > > > > 2017-04-25 14:13:00,711
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > > >
> > > > > 2017-04-25 14:13:00,711
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
> http://public-repo-1.
>
> > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> > > ['create'],
> > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> 'HDP-UTILS',
> > > > > 'mirror_list': None}
> > > > >
> > > > > 2017-04-25 14:13:00,717
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > > InlineTemplate(...)}
> > > > >
> > > > > 2017-04-25 14:13:00,717
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > > > >
> > > > > 2017-04-25 14:13:00,718
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Package['unzip']
> > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:00,846
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > > >
> > > > > 2017-04-25 14:13:03,413
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Package['curl']
> > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:03,429
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > > > installation of existing package curl
> > > > >
> > > > > 2017-04-25 14:13:03,430
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > > > 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:03,451
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
hdp-select')
> > > > >
> > > > > 2017-04-25 14:13:04,948
> > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES
> > > Data
> > > > > Node
> > > > >
> > > > > 2017-04-25 14:13:04,952
> > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> > > False,
> > > > > 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:05,086
> > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > > > elasticsearch-2.3.3')
> > > > >
> > > > >
> > > > >
> > > > > Command failed after 1 tries
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
> ottobackwards@gmail.com)
>
> > > > > wrote:
> > > > >
> > > > > Yes, I’m going to run through again and get the logs. The basic
> issue
> > > > is,
> > > > > no repo tab -> if I should have one,
> > > > > and the ES yum command failing with nothing to do, which stops
> > > deployment
> > > > > on all the nodes.
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > > michael.miklavcic@gmail.com) wrote:
> > > > >
> > > > > I've been doing an internal hwx deployment (HCP) recently for
> testing
> > > > > purposes. I have some ES tweaks to work out, but other than that
it
> > > has
> > > > > been running smoothly. Can you share any log info or screenshots?
> > > > >
> > > > > Mike
> > > > >
> > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Hi,
> > > > > > Should I see the repo tab in the ambari wizard for metron at
this
> > > > point?
> > > > > > Asking for a friend who can’t get the wizard to work :)
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> >
>
--
Jon
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
Getting the same error in ES as before from what I can see
On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwards@gmail.com) wrote:
I still can’t keep ES running, it dies.
Also, I’m getting exceptions from enrichment trying to restart when the
topology wasn’t running.
Is there some coordination in this work? Should I just log jiras?
If someone is already sussing out ES, we can sync up
On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com) wrote:
I'm also interested to know why that's important at such a small scale.
Jon
On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com> wrote:
> I am following
>
>
https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html
> I DO have master and data node together.
> Why is that a problem?
>
> I will try again with master and data node separate.
>
>
> On April 26, 2017 at 10:41:59, Michael Miklavcic (
> michael.miklavcic@gmail.com) wrote:
>
> Hey Otto,
>
> How do you have the ES nodes configured? For a base install I would setup
1
> master (NOT as data node) and 2 data nodes (NOT on the same node as the
> master). This is the install configuration I got working. You can also
> modify some configuration properties around master node as data node,
index
> replicas, and gateway recovery to get it working differently, but this is
> what will work OOTB with the default config settings from the mpack. If
> you've already setup a master node and a data node on the same host, we'll
> need to re-install.
>
> Mike
>
>
> On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > File
> > "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line
> > 293, in _call
> >
> > raise ExecutionFailed(err_msg, code, out, err)
> >
> > ExecutionFailed: Execution of 'service elasticsearch status' returned 3.
> ●
> > elasticsearch.service - Elasticsearch
> >
> > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> > disabled;
> > vendor preset: disabled)
> >
> > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58 EDT
> > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
> >
> > Docs: http://www.elastic.co
> >
> > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch
> > -Des.pidfile=${PID_DIR}/elasticsearch.pid -Des.default.path.home=${ES_
> > HOME}
> > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR}
> > -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
> >
> > Process: 16819
> > ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec
> > (code=exited, status=0/SUCCESS)
> >
> > Main PID: 16821 (code=exited, status=1/FAILURE)
> >
> >
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.common.settings.Settings$Builder.
> > loadFromStream(Settings.java:1080)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.common.settings.Settings$Builder.
> > loadFromPath(Settings.java:1067)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.node.internal.InternalSettingsPreparer.
> > prepareEnvironment(InternalSettingsPreparer.java:88)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> >
org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: Refer
> > to
> > the log for complete error details.
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > elasticsearch.service: main process exited, code=exited,
status=1/FAILURE
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> > elasticsearch.service entered failed state.
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > elasticsearch.service failed.
> >
> > INFO 2017-04-26 08:55:19,962
> > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
> >
> >
> > On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Can you describe what ES issues you are working on?
> > Fixing the repos got everything installed, but my ES components don’t
> stay
> > running.
> > I need to harvest the errors.
> >
> >
> >
> > On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Nm. sorry. I fixed it.
> >
> >
> >
> > On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Ok, now I see the repos in the ‘pick version’ screen, but it is erring
on
> > the f://localrepo
> > even though the folder exists, there is no repodata/repomd.xml.
> >
> > What is the command to create a local repo?
> >
> >
> >
> > On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > I was going by the HW community page.
> >
> > Ok, Let me try it
> >
> >
> >
> > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com) wrote:
> >
> > That would do it. It requires 2.4.2+. I would have sworn I put that in
> the
> > README, but I must have only annotated the PR. :(
> >
> > I'll get that in asap.
> >
> > -D...
> >
> >
> > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > 2.4.1 as a matter of fact
> > >
> > >
> > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com)
> wrote:
> > >
> > > Any chance they're running Ambari < 2.4.2?
> > >
> > > -D...
> > >
> > >
> > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > > michael.miklavcic@gmail.com> wrote:
> > >
> > > > Hey Otto, I don't have wizard screenshots in front of me right now
to
> > > say
> > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't run
> into
> > > > anything like that, but next time I run through the install I can
> take
> > > > another look.
> > > >
> > > > https://github.com/apache/incubator-metron/blob/master/
> > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> > > >
> > > >
> > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
> ottobackwards@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > stderr:
> > > > >
> > > > > Traceback (most recent call last):
> > > > >
> > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > > > >
> > > > > Elasticsearch().execute()
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > libraries/script/script.py",
> > > > > line 280, in execute
> > > > >
> > > > > method(env)
> > > > >
> > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > > > >
> > > > > self.install_packages(env)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > libraries/script/script.py",
> > > > > line 567, in install_packages
> > > > >
> > > > > retry_count=agent_stack_retry_count)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/base.py",
> > > > > line 155, in __init__
> > > > >
> > > > > self.env.run()
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/environment.py",
> > > > > line 160, in run
> > > > >
> > > > > self.run_action(resource, action)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/environment.py",
> > > > > line 124, in run_action
> > > > >
> > > > > provider_action()
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/__init__.py", line 54, in action_install
> > > > >
> > > > > self.install_package(package_name, self.resource.use_repos,
> > > > > self.resource.skip_repos)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/yumrpm.py", line 49, in install_package
> > > > >
> > > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > > logoutput=self.get_logoutput())
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/__init__.py", line 83, in
> > > > checked_call_with_retries
> > > > >
> > > > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/__init__.py", line 91, in
_call_with_retries
> > > > >
> > > > > code, out = func(cmd, **kwargs)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 71, in inner
> > > > >
> > > > > result = function(command, **kwargs)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 93, in checked_call
> > > > >
> > > > > tries=tries, try_sleep=try_sleep)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 141, in _call_wrapper
> > > > >
> > > > > result = _call(command, **kwargs_copy)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 294, in _call
> > > > >
> > > > > raise Fail(err_msg)
> > > > >
> > > > > resource_management.core.exceptions.Fail: Execution of
> '/usr/bin/yum
> > > -d
> > > > 0
> > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to
> do
> > > > >
> > > > > stdout:
> > > > >
> > > > > 2017-04-25 14:12:48,669
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using
> hadoop
> > > > conf
> > > > > dir: /usr/hdp/current/hadoop-client/conf
> > > > >
> > > > > 2017-04-25 14:12:48,671
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['metron']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,673
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['metron']
> > > > >
> > > > > 2017-04-25 14:12:48,740
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['livy']
> > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,741
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['livy']
> > > > >
> > > > > 2017-04-25 14:12:48,774
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['elasticsearch'] {}
> > > > >
> > > > > 2017-04-25 14:12:48,775
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['elasticsearch']
> > > > >
> > > > > 2017-04-25 14:12:48,812
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['spark']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,812
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['spark']
> > > > >
> > > > > 2017-04-25 14:12:48,848
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['zeppelin']
> > > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,849
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['zeppelin']
> > > > >
> > > > > 2017-04-25 14:12:48,884
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['hadoop']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,885
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['hadoop']
> > > > >
> > > > > 2017-04-25 14:12:48,921
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['kibana']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,922
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['kibana']
> > > > >
> > > > > 2017-04-25 14:12:48,959
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['users']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,960
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> User['hive']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:48,961
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> user
> > > > > User['hive']
> > > > >
> > > > > 2017-04-25 14:12:49,227
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['storm']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,227
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['storm']
> > > > >
> > > > > 2017-04-25 14:12:49,388
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['zookeeper']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,388
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['zookeeper']
> > > > >
> > > > > 2017-04-25 14:12:49,518
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
True,
> > > > > 'groups': [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,518
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['infra-solr']
> > > > >
> > > > > 2017-04-25 14:12:49,578
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['ams']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,578
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['ams']
> > > > >
> > > > > 2017-04-25 14:12:49,625
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['tez']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'users']}
> > > > >
> > > > > 2017-04-25 14:12:49,626
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['tez']
> > > > >
> > > > > 2017-04-25 14:12:49,660
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['zeppelin']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,660
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['zeppelin']
> > > > >
> > > > > 2017-04-25 14:12:49,705
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['metron']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,705
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['metron']
> > > > >
> > > > > 2017-04-25 14:12:49,760
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['livy']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,760
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['livy']
> > > > >
> > > > > 2017-04-25 14:12:49,803
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> > True,
> > > > > 'groups': [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,804
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['elasticsearch']
> > > > >
> > > > > 2017-04-25 14:12:49,837
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['spark']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,837
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['spark']
> > > > >
> > > > > 2017-04-25 14:12:49,885
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['ambari-qa']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'users']}
> > > > >
> > > > > 2017-04-25 14:12:49,885
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['ambari-qa']
> > > > >
> > > > > 2017-04-25 14:12:49,928
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['flume']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,928
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['flume']
> > > > >
> > > > > 2017-04-25 14:12:49,972
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['kafka']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,973
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['kafka']
> > > > >
> > > > > 2017-04-25 14:12:50,025
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hdfs']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,026
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['hdfs']
> > > > >
> > > > > 2017-04-25 14:12:50,075
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['yarn']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,075
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['yarn']
> > > > >
> > > > > 2017-04-25 14:12:50,125
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['kibana']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,126
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['kibana']
> > > > >
> > > > > 2017-04-25 14:12:50,175
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['mapred']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,176
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['mapred']
> > > > >
> > > > > 2017-04-25 14:12:50,223
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['hbase']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,224
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['hbase']
> > > > >
> > > > > 2017-04-25 14:12:50,271
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hcat']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,272
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['hcat']
> > > > >
> > > > > 2017-04-25 14:12:50,320
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > >
> > > > > 2017-04-25 14:12:50,333
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
> > > exist
> > > > >
> > > > > 2017-04-25 14:12:50,334
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to
> > 555
> > > > >
> > > > > 2017-04-25 14:12:50,335
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
> > > $(id
> > > > -u
> > > > > ambari-qa) -gt 1000) || (false)'}
> > > > >
> > > > > 2017-04-25 14:12:50,364
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > > > >
> > > > > 2017-04-25 14:12:50,365
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
> > > True,
> > > > > 'mode': 0775, 'cd_access': 'a'}
> > > > >
> > > > > 2017-04-25 14:12:50,366
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,366
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > owner
> > > > > for /tmp/hbase-hbase from 0 to hbase
> > > > >
> > > > > 2017-04-25 14:12:50,366
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > permission for /tmp/hbase-hbase from 755 to 775
> > > > >
> > > > > 2017-04-25 14:12:50,368
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > >
> > > > > 2017-04-25 14:12:50,369
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > >
> > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> > >
> > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > > >
> > > > > 2017-04-25 14:12:50,393
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > >
> > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> > >
> > > > > due to not_if
> > > > >
> > > > > 2017-04-25 14:12:50,394
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Group['hdfs']
> > > {}
> > > > >
> > > > > 2017-04-25 14:12:50,395
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> group
> > > > > Group['hdfs']
> > > > >
> > > > > 2017-04-25 14:12:50,430
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hdfs']
> > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > > > >
> > > > > 2017-04-25 14:12:50,431
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
> > > user
> > > > > hdfs
> > > > >
> > > > > 2017-04-25 14:12:50,474
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > > > >
> > > > > 2017-04-25 14:12:50,474
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > > >
> > > > > 2017-04-25 14:12:50,474
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,475
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > > {'owner':
> > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > > >
> > > > > 2017-04-25 14:12:50,476
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
> > tmpdir']
> > >
> > > > > since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,476
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > owner
> > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
> > > > >
> > > > > 2017-04-25 14:12:50,476
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > group
> > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
> hadoop
> > > > >
> > > > > 2017-04-25 14:12:50,477
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
from
> > > 755
> > > > > to 1777
> > > > >
> > > > > 2017-04-25 14:12:50,477
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > > {'create_parents': True}
> > > > >
> > > > > 2017-04-25 14:12:50,478
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > since
> > > > > it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,479
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > > {'content':
> > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > > >
> > > > > 2017-04-25 14:12:50,504
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Downloading
> > > the
> > > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > > linux-x64.tar.gz
> > > > >
> > > > > 2017-04-25 14:12:53,793
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > Directory['/usr/jdk64'] {}
> > > > >
> > > > > 2017-04-25 14:12:53,793
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:53,794
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > Execute[('chmod',
> > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > > >
> > > > > 2017-04-25 14:12:53,824
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> Execute['cd
> > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> > ambari-sudo.sh
> > > cp
> > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > > > >
> > > > > 2017-04-25 14:13:00,439
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > > > ['delete']}
> > > > >
> > > > > 2017-04-25 14:13:00,440
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
and
> > > all
> > > > > its content
> > > > >
> > > > > 2017-04-25 14:13:00,620
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
'cd_access':
> > > 'a'}
> > > > >
> > > > > 2017-04-25 14:13:00,621
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Execute[('chmod',
> > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > > >
> > > > > 2017-04-25 14:13:00,682
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Initializing
> > > 2
> > > > > repositories
> > > > >
> > > > > 2017-04-25 14:13:00,683
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > > ['create'],
> > > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > > > 'mirror_list': None}
> > > > >
> > > > > 2017-04-25 14:13:00,709
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
> > > > >
> > > > > 2017-04-25 14:13:00,711
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > > >
> > > > > 2017-04-25 14:13:00,711
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
> http://public-repo-1.
>
> > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> > > ['create'],
> > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> 'HDP-UTILS',
> > > > > 'mirror_list': None}
> > > > >
> > > > > 2017-04-25 14:13:00,717
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > > InlineTemplate(...)}
> > > > >
> > > > > 2017-04-25 14:13:00,717
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > > > >
> > > > > 2017-04-25 14:13:00,718
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Package['unzip']
> > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:00,846
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > > >
> > > > > 2017-04-25 14:13:03,413
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Package['curl']
> > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:03,429
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > > > installation of existing package curl
> > > > >
> > > > > 2017-04-25 14:13:03,430
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > > > 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:03,451
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
hdp-select')
> > > > >
> > > > > 2017-04-25 14:13:04,948
> > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES
> > > Data
> > > > > Node
> > > > >
> > > > > 2017-04-25 14:13:04,952
> > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> > > False,
> > > > > 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:05,086
> > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > > > elasticsearch-2.3.3')
> > > > >
> > > > >
> > > > >
> > > > > Command failed after 1 tries
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
> ottobackwards@gmail.com)
>
> > > > > wrote:
> > > > >
> > > > > Yes, I’m going to run through again and get the logs. The basic
> issue
> > > > is,
> > > > > no repo tab -> if I should have one,
> > > > > and the ES yum command failing with nothing to do, which stops
> > > deployment
> > > > > on all the nodes.
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > > michael.miklavcic@gmail.com) wrote:
> > > > >
> > > > > I've been doing an internal hwx deployment (HCP) recently for
> testing
> > > > > purposes. I have some ES tweaks to work out, but other than that
it
> > > has
> > > > > been running smoothly. Can you share any log info or screenshots?
> > > > >
> > > > > Mike
> > > > >
> > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Hi,
> > > > > > Should I see the repo tab in the ambari wizard for metron at
this
> > > > point?
> > > > > > Asking for a friend who can’t get the wizard to work :)
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> >
>
--
Jon
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
I still can’t keep ES running, it dies.
Also, I’m getting exceptions from enrichment trying to restart when the
topology wasn’t running.
Is there some coordination in this work? Should I just log jiras?
If someone is already sussing out ES, we can sync up
On April 26, 2017 at 11:35:03, Zeolla@GMail.com (zeolla@gmail.com) wrote:
I'm also interested to know why that's important at such a small scale.
Jon
On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com> wrote:
> I am following
>
>
https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html
> I DO have master and data node together.
> Why is that a problem?
>
> I will try again with master and data node separate.
>
>
> On April 26, 2017 at 10:41:59, Michael Miklavcic (
> michael.miklavcic@gmail.com) wrote:
>
> Hey Otto,
>
> How do you have the ES nodes configured? For a base install I would setup
1
> master (NOT as data node) and 2 data nodes (NOT on the same node as the
> master). This is the install configuration I got working. You can also
> modify some configuration properties around master node as data node,
index
> replicas, and gateway recovery to get it working differently, but this is
> what will work OOTB with the default config settings from the mpack. If
> you've already setup a master node and a data node on the same host,
we'll
> need to re-install.
>
> Mike
>
>
> On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > File
> > "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line
> > 293, in _call
> >
> > raise ExecutionFailed(err_msg, code, out, err)
> >
> > ExecutionFailed: Execution of 'service elasticsearch status' returned
3.
> ●
> > elasticsearch.service - Elasticsearch
> >
> > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> > disabled;
> > vendor preset: disabled)
> >
> > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58 EDT
> > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
> >
> > Docs: http://www.elastic.co
> >
> > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch
> > -Des.pidfile=${PID_DIR}/elasticsearch.pid -Des.default.path.home=${ES_
> > HOME}
> > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR}
> > -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
> >
> > Process: 16819
> >
ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec
> > (code=exited, status=0/SUCCESS)
> >
> > Main PID: 16821 (code=exited, status=1/FAILURE)
> >
> >
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.common.settings.Settings$Builder.
> > loadFromStream(Settings.java:1080)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.common.settings.Settings$Builder.
> > loadFromPath(Settings.java:1067)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.node.internal.InternalSettingsPreparer.
> > prepareEnvironment(InternalSettingsPreparer.java:88)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> >
org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]:
Refer
> > to
> > the log for complete error details.
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > elasticsearch.service: main process exited, code=exited,
status=1/FAILURE
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> > elasticsearch.service entered failed state.
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > elasticsearch.service failed.
> >
> > INFO 2017-04-26 08:55:19,962
> > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
> >
> >
> > On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Can you describe what ES issues you are working on?
> > Fixing the repos got everything installed, but my ES components don’t
> stay
> > running.
> > I need to harvest the errors.
> >
> >
> >
> > On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Nm. sorry. I fixed it.
> >
> >
> >
> > On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Ok, now I see the repos in the ‘pick version’ screen, but it is erring
on
> > the f://localrepo
> > even though the folder exists, there is no repodata/repomd.xml.
> >
> > What is the command to create a local repo?
> >
> >
> >
> > On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > I was going by the HW community page.
> >
> > Ok, Let me try it
> >
> >
> >
> > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com) wrote:
> >
> > That would do it. It requires 2.4.2+. I would have sworn I put that in
> the
> > README, but I must have only annotated the PR. :(
> >
> > I'll get that in asap.
> >
> > -D...
> >
> >
> > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > 2.4.1 as a matter of fact
> > >
> > >
> > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com)
> wrote:
> > >
> > > Any chance they're running Ambari < 2.4.2?
> > >
> > > -D...
> > >
> > >
> > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > > michael.miklavcic@gmail.com> wrote:
> > >
> > > > Hey Otto, I don't have wizard screenshots in front of me right now
to
> > > say
> > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't run
> into
> > > > anything like that, but next time I run through the install I can
> take
> > > > another look.
> > > >
> > > > https://github.com/apache/incubator-metron/blob/master/
> > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> > > >
> > > >
> > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
> ottobackwards@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > stderr:
> > > > >
> > > > > Traceback (most recent call last):
> > > > >
> > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > > > >
> > > > > Elasticsearch().execute()
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > libraries/script/script.py",
> > > > > line 280, in execute
> > > > >
> > > > > method(env)
> > > > >
> > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > > > >
> > > > > self.install_packages(env)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > libraries/script/script.py",
> > > > > line 567, in install_packages
> > > > >
> > > > > retry_count=agent_stack_retry_count)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/base.py",
> > > > > line 155, in __init__
> > > > >
> > > > > self.env.run()
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/environment.py",
> > > > > line 160, in run
> > > > >
> > > > > self.run_action(resource, action)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/environment.py",
> > > > > line 124, in run_action
> > > > >
> > > > > provider_action()
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/__init__.py", line 54, in action_install
> > > > >
> > > > > self.install_package(package_name, self.resource.use_repos,
> > > > > self.resource.skip_repos)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/yumrpm.py", line 49, in install_package
> > > > >
> > > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > > logoutput=self.get_logoutput())
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/__init__.py", line 83, in
> > > > checked_call_with_retries
> > > > >
> > > > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/__init__.py", line 91, in
_call_with_retries
> > > > >
> > > > > code, out = func(cmd, **kwargs)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 71, in inner
> > > > >
> > > > > result = function(command, **kwargs)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 93, in checked_call
> > > > >
> > > > > tries=tries, try_sleep=try_sleep)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 141, in _call_wrapper
> > > > >
> > > > > result = _call(command, **kwargs_copy)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 294, in _call
> > > > >
> > > > > raise Fail(err_msg)
> > > > >
> > > > > resource_management.core.exceptions.Fail: Execution of
> '/usr/bin/yum
> > > -d
> > > > 0
> > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing
to
> do
> > > > >
> > > > > stdout:
> > > > >
> > > > > 2017-04-25 14:12:48,669
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using
> hadoop
> > > > conf
> > > > > dir: /usr/hdp/current/hadoop-client/conf
> > > > >
> > > > > 2017-04-25 14:12:48,671
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['metron']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,673
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['metron']
> > > > >
> > > > > 2017-04-25 14:12:48,740
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['livy']
> > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,741
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['livy']
> > > > >
> > > > > 2017-04-25 14:12:48,774
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['elasticsearch'] {}
> > > > >
> > > > > 2017-04-25 14:12:48,775
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['elasticsearch']
> > > > >
> > > > > 2017-04-25 14:12:48,812
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['spark']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,812
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['spark']
> > > > >
> > > > > 2017-04-25 14:12:48,848
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['zeppelin']
> > > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,849
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['zeppelin']
> > > > >
> > > > > 2017-04-25 14:12:48,884
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['hadoop']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,885
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['hadoop']
> > > > >
> > > > > 2017-04-25 14:12:48,921
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['kibana']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,922
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['kibana']
> > > > >
> > > > > 2017-04-25 14:12:48,959
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['users']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,960
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> User['hive']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:48,961
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> user
> > > > > User['hive']
> > > > >
> > > > > 2017-04-25 14:12:49,227
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['storm']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,227
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['storm']
> > > > >
> > > > > 2017-04-25 14:12:49,388
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['zookeeper']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,388
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['zookeeper']
> > > > >
> > > > > 2017-04-25 14:12:49,518
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
True,
> > > > > 'groups': [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,518
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['infra-solr']
> > > > >
> > > > > 2017-04-25 14:12:49,578
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['ams']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,578
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['ams']
> > > > >
> > > > > 2017-04-25 14:12:49,625
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['tez']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'users']}
> > > > >
> > > > > 2017-04-25 14:12:49,626
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['tez']
> > > > >
> > > > > 2017-04-25 14:12:49,660
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['zeppelin']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,660
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['zeppelin']
> > > > >
> > > > > 2017-04-25 14:12:49,705
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['metron']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,705
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['metron']
> > > > >
> > > > > 2017-04-25 14:12:49,760
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['livy']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,760
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['livy']
> > > > >
> > > > > 2017-04-25 14:12:49,803
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> > True,
> > > > > 'groups': [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,804
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['elasticsearch']
> > > > >
> > > > > 2017-04-25 14:12:49,837
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['spark']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,837
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['spark']
> > > > >
> > > > > 2017-04-25 14:12:49,885
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['ambari-qa']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'users']}
> > > > >
> > > > > 2017-04-25 14:12:49,885
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['ambari-qa']
> > > > >
> > > > > 2017-04-25 14:12:49,928
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['flume']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,928
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['flume']
> > > > >
> > > > > 2017-04-25 14:12:49,972
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['kafka']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,973
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['kafka']
> > > > >
> > > > > 2017-04-25 14:12:50,025
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hdfs']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,026
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['hdfs']
> > > > >
> > > > > 2017-04-25 14:12:50,075
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['yarn']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,075
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['yarn']
> > > > >
> > > > > 2017-04-25 14:12:50,125
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['kibana']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,126
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['kibana']
> > > > >
> > > > > 2017-04-25 14:12:50,175
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['mapred']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,176
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['mapred']
> > > > >
> > > > > 2017-04-25 14:12:50,223
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['hbase']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,224
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['hbase']
> > > > >
> > > > > 2017-04-25 14:12:50,271
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hcat']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,272
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['hcat']
> > > > >
> > > > > 2017-04-25 14:12:50,320
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > >
> > > > > 2017-04-25 14:12:50,333
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
> > > exist
> > > > >
> > > > > 2017-04-25 14:12:50,334
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to
> > 555
> > > > >
> > > > > 2017-04-25 14:12:50,335
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
> > > $(id
> > > > -u
> > > > > ambari-qa) -gt 1000) || (false)'}
> > > > >
> > > > > 2017-04-25 14:12:50,364
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > > > >
> > > > > 2017-04-25 14:12:50,365
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
'create_parents':
> > > True,
> > > > > 'mode': 0775, 'cd_access': 'a'}
> > > > >
> > > > > 2017-04-25 14:12:50,366
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,366
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > owner
> > > > > for /tmp/hbase-hbase from 0 to hbase
> > > > >
> > > > > 2017-04-25 14:12:50,366
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > permission for /tmp/hbase-hbase from 755 to 775
> > > > >
> > > > > 2017-04-25 14:12:50,368
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > >
> > > > > 2017-04-25 14:12:50,369
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > >
> > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> > >
> > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > > >
> > > > > 2017-04-25 14:12:50,393
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > >
> > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> > >
> > > > > due to not_if
> > > > >
> > > > > 2017-04-25 14:12:50,394
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Group['hdfs']
> > > {}
> > > > >
> > > > > 2017-04-25 14:12:50,395
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> group
> > > > > Group['hdfs']
> > > > >
> > > > > 2017-04-25 14:12:50,430
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hdfs']
> > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > > > >
> > > > > 2017-04-25 14:12:50,431
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
> > > user
> > > > > hdfs
> > > > >
> > > > > 2017-04-25 14:12:50,474
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > > > >
> > > > > 2017-04-25 14:12:50,474
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > > >
> > > > > 2017-04-25 14:12:50,474
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,475
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > > {'owner':
> > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > > >
> > > > > 2017-04-25 14:12:50,476
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
> > tmpdir']
> > >
> > > > > since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,476
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > owner
> > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
hdfs
> > > > >
> > > > > 2017-04-25 14:12:50,476
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > group
> > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
> hadoop
> > > > >
> > > > > 2017-04-25 14:12:50,477
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
from
> > > 755
> > > > > to 1777
> > > > >
> > > > > 2017-04-25 14:12:50,477
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > > {'create_parents': True}
> > > > >
> > > > > 2017-04-25 14:12:50,478
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory
Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > since
> > > > > it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,479
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > > {'content':
> > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > > >
> > > > > 2017-04-25 14:12:50,504
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Downloading
> > > the
> > > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > > linux-x64.tar.gz
> > > > >
> > > > > 2017-04-25 14:12:53,793
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > Directory['/usr/jdk64'] {}
> > > > >
> > > > > 2017-04-25 14:12:53,793
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:53,794
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > Execute[('chmod',
> > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > > >
> > > > > 2017-04-25 14:12:53,824
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> Execute['cd
> > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> > ambari-sudo.sh
> > > cp
> > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > > > >
> > > > > 2017-04-25 14:13:00,439
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > > > ['delete']}
> > > > >
> > > > > 2017-04-25 14:13:00,440
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS']
and
> > > all
> > > > > its content
> > > > >
> > > > > 2017-04-25 14:13:00,620
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755,
'cd_access':
> > > 'a'}
> > > > >
> > > > > 2017-04-25 14:13:00,621
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Execute[('chmod',
> > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > > >
> > > > > 2017-04-25 14:13:00,682
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Initializing
> > > 2
> > > > > repositories
> > > > >
> > > > > 2017-04-25 14:13:00,683
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > > ['create'],
> > > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
endif
> > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > > > 'mirror_list': None}
> > > > >
> > > > > 2017-04-25 14:13:00,709
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > File['/etc/yum.repos.d/HDP.repo'] {'content':
InlineTemplate(...)}
> > > > >
> > > > > 2017-04-25 14:13:00,711
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > > >
> > > > > 2017-04-25 14:13:00,711
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
> http://public-repo-1.
>
> > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> > > ['create'],
> > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{%
endif
> > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> 'HDP-UTILS',
> > > > > 'mirror_list': None}
> > > > >
> > > > > 2017-04-25 14:13:00,717
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > > InlineTemplate(...)}
> > > > >
> > > > > 2017-04-25 14:13:00,717
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > > > >
> > > > > 2017-04-25 14:13:00,718
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Package['unzip']
> > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:00,846
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
Installing
> > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > > >
> > > > > 2017-04-25 14:13:03,413
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Package['curl']
> > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:03,429
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > > > installation of existing package curl
> > > > >
> > > > > 2017-04-25 14:13:03,430
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > > > 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:03,451
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
Installing
> > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install
hdp-select')
> > > > >
> > > > > 2017-04-25 14:13:04,948
> > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install
ES
> > > Data
> > > > > Node
> > > > >
> > > > > 2017-04-25 14:13:04,952
> > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> > > False,
> > > > > 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:05,086
> > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> -
Installing
> > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > > > elasticsearch-2.3.3')
> > > > >
> > > > >
> > > > >
> > > > > Command failed after 1 tries
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
> ottobackwards@gmail.com)
>
> > > > > wrote:
> > > > >
> > > > > Yes, I’m going to run through again and get the logs. The basic
> issue
> > > > is,
> > > > > no repo tab -> if I should have one,
> > > > > and the ES yum command failing with nothing to do, which stops
> > > deployment
> > > > > on all the nodes.
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > > michael.miklavcic@gmail.com) wrote:
> > > > >
> > > > > I've been doing an internal hwx deployment (HCP) recently for
> testing
> > > > > purposes. I have some ES tweaks to work out, but other than that
it
> > > has
> > > > > been running smoothly. Can you share any log info or screenshots?
> > > > >
> > > > > Mike
> > > > >
> > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Hi,
> > > > > > Should I see the repo tab in the ambari wizard for metron at
this
> > > > point?
> > > > > > Asking for a friend who can’t get the wizard to work :)
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> >
>
--
Jon
Re: Ambari Wizard: Repo Tab
Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
I'm also interested to know why that's important at such a small scale.
Jon
On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ot...@gmail.com> wrote:
> I am following
>
> https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html
> I DO have master and data node together.
> Why is that a problem?
>
> I will try again with master and data node separate.
>
>
> On April 26, 2017 at 10:41:59, Michael Miklavcic (
> michael.miklavcic@gmail.com) wrote:
>
> Hey Otto,
>
> How do you have the ES nodes configured? For a base install I would setup 1
> master (NOT as data node) and 2 data nodes (NOT on the same node as the
> master). This is the install configuration I got working. You can also
> modify some configuration properties around master node as data node, index
> replicas, and gateway recovery to get it working differently, but this is
> what will work OOTB with the default config settings from the mpack. If
> you've already setup a master node and a data node on the same host, we'll
> need to re-install.
>
> Mike
>
>
> On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > File
> > "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line
> > 293, in _call
> >
> > raise ExecutionFailed(err_msg, code, out, err)
> >
> > ExecutionFailed: Execution of 'service elasticsearch status' returned 3.
> ●
> > elasticsearch.service - Elasticsearch
> >
> > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> > disabled;
> > vendor preset: disabled)
> >
> > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58 EDT
> > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
> >
> > Docs: http://www.elastic.co
> >
> > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch
> > -Des.pidfile=${PID_DIR}/elasticsearch.pid -Des.default.path.home=${ES_
> > HOME}
> > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR}
> > -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
> >
> > Process: 16819
> > ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec
> > (code=exited, status=0/SUCCESS)
> >
> > Main PID: 16821 (code=exited, status=1/FAILURE)
> >
> >
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.common.settings.Settings$Builder.
> > loadFromStream(Settings.java:1080)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.common.settings.Settings$Builder.
> > loadFromPath(Settings.java:1067)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.node.internal.InternalSettingsPreparer.
> > prepareEnvironment(InternalSettingsPreparer.java:88)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> > org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: Refer
> > to
> > the log for complete error details.
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > elasticsearch.service: main process exited, code=exited, status=1/FAILURE
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> > elasticsearch.service entered failed state.
> >
> > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> > elasticsearch.service failed.
> >
> > INFO 2017-04-26 08:55:19,962
> > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
> >
> >
> > On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Can you describe what ES issues you are working on?
> > Fixing the repos got everything installed, but my ES components don’t
> stay
> > running.
> > I need to harvest the errors.
> >
> >
> >
> > On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Nm. sorry. I fixed it.
> >
> >
> >
> > On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Ok, now I see the repos in the ‘pick version’ screen, but it is erring on
> > the f://localrepo
> > even though the folder exists, there is no repodata/repomd.xml.
> >
> > What is the command to create a local repo?
> >
> >
> >
> > On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > I was going by the HW community page.
> >
> > Ok, Let me try it
> >
> >
> >
> > On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com) wrote:
> >
> > That would do it. It requires 2.4.2+. I would have sworn I put that in
> the
> > README, but I must have only annotated the PR. :(
> >
> > I'll get that in asap.
> >
> > -D...
> >
> >
> > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > 2.4.1 as a matter of fact
> > >
> > >
> > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com)
> wrote:
> > >
> > > Any chance they're running Ambari < 2.4.2?
> > >
> > > -D...
> > >
> > >
> > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > > michael.miklavcic@gmail.com> wrote:
> > >
> > > > Hey Otto, I don't have wizard screenshots in front of me right now to
> > > say
> > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't run
> into
> > > > anything like that, but next time I run through the install I can
> take
> > > > another look.
> > > >
> > > > https://github.com/apache/incubator-metron/blob/master/
> > > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> > > >
> > > >
> > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <
> ottobackwards@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > stderr:
> > > > >
> > > > > Traceback (most recent call last):
> > > > >
> > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > > > >
> > > > > Elasticsearch().execute()
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > libraries/script/script.py",
> > > > > line 280, in execute
> > > > >
> > > > > method(env)
> > > > >
> > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > > > >
> > > > > self.install_packages(env)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > libraries/script/script.py",
> > > > > line 567, in install_packages
> > > > >
> > > > > retry_count=agent_stack_retry_count)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/base.py",
> > > > > line 155, in __init__
> > > > >
> > > > > self.env.run()
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/environment.py",
> > > > > line 160, in run
> > > > >
> > > > > self.run_action(resource, action)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/environment.py",
> > > > > line 124, in run_action
> > > > >
> > > > > provider_action()
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/__init__.py", line 54, in action_install
> > > > >
> > > > > self.install_package(package_name, self.resource.use_repos,
> > > > > self.resource.skip_repos)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/yumrpm.py", line 49, in install_package
> > > > >
> > > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > > logoutput=self.get_logoutput())
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/__init__.py", line 83, in
> > > > checked_call_with_retries
> > > > >
> > > > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > > core/providers/package/__init__.py", line 91, in _call_with_retries
> > > > >
> > > > > code, out = func(cmd, **kwargs)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 71, in inner
> > > > >
> > > > > result = function(command, **kwargs)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 93, in checked_call
> > > > >
> > > > > tries=tries, try_sleep=try_sleep)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 141, in _call_wrapper
> > > > >
> > > > > result = _call(command, **kwargs_copy)
> > > > >
> > > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/shell.py",
> > > > > line 294, in _call
> > > > >
> > > > > raise Fail(err_msg)
> > > > >
> > > > > resource_management.core.exceptions.Fail: Execution of
> '/usr/bin/yum
> > > -d
> > > > 0
> > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to
> do
> > > > >
> > > > > stdout:
> > > > >
> > > > > 2017-04-25 14:12:48,669
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using
> hadoop
> > > > conf
> > > > > dir: /usr/hdp/current/hadoop-client/conf
> > > > >
> > > > > 2017-04-25 14:12:48,671
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['metron']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,673
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['metron']
> > > > >
> > > > > 2017-04-25 14:12:48,740
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['livy']
> > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,741
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['livy']
> > > > >
> > > > > 2017-04-25 14:12:48,774
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > > Group['elasticsearch'] {}
> > > > >
> > > > > 2017-04-25 14:12:48,775
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['elasticsearch']
> > > > >
> > > > > 2017-04-25 14:12:48,812
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['spark']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,812
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['spark']
> > > > >
> > > > > 2017-04-25 14:12:48,848
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['zeppelin']
> > > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,849
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['zeppelin']
> > > > >
> > > > > 2017-04-25 14:12:48,884
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['hadoop']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,885
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['hadoop']
> > > > >
> > > > > 2017-04-25 14:12:48,921
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['kibana']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,922
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> group
> > > > > Group['kibana']
> > > > >
> > > > > 2017-04-25 14:12:48,959
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['users']
> > > > {}
> > > > >
> > > > > 2017-04-25 14:12:48,960
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> User['hive']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:48,961
> > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
> user
> > > > > User['hive']
> > > > >
> > > > > 2017-04-25 14:12:49,227
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['storm']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,227
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['storm']
> > > > >
> > > > > 2017-04-25 14:12:49,388
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['zookeeper']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,388
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['zookeeper']
> > > > >
> > > > > 2017-04-25 14:12:49,518
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > > > > 'groups': [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,518
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['infra-solr']
> > > > >
> > > > > 2017-04-25 14:12:49,578
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['ams']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,578
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['ams']
> > > > >
> > > > > 2017-04-25 14:12:49,625
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['tez']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'users']}
> > > > >
> > > > > 2017-04-25 14:12:49,626
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['tez']
> > > > >
> > > > > 2017-04-25 14:12:49,660
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['zeppelin']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,660
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['zeppelin']
> > > > >
> > > > > 2017-04-25 14:12:49,705
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['metron']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,705
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['metron']
> > > > >
> > > > > 2017-04-25 14:12:49,760
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['livy']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,760
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['livy']
> > > > >
> > > > > 2017-04-25 14:12:49,803
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> > True,
> > > > > 'groups': [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,804
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['elasticsearch']
> > > > >
> > > > > 2017-04-25 14:12:49,837
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['spark']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,837
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['spark']
> > > > >
> > > > > 2017-04-25 14:12:49,885
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['ambari-qa']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'users']}
> > > > >
> > > > > 2017-04-25 14:12:49,885
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['ambari-qa']
> > > > >
> > > > > 2017-04-25 14:12:49,928
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['flume']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,928
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['flume']
> > > > >
> > > > > 2017-04-25 14:12:49,972
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['kafka']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:49,973
> > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding
> user
> > > > > User['kafka']
> > > > >
> > > > > 2017-04-25 14:12:50,025
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hdfs']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,026
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['hdfs']
> > > > >
> > > > > 2017-04-25 14:12:50,075
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['yarn']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,075
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['yarn']
> > > > >
> > > > > 2017-04-25 14:12:50,125
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['kibana']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,126
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['kibana']
> > > > >
> > > > > 2017-04-25 14:12:50,175
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > User['mapred']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,176
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['mapred']
> > > > >
> > > > > 2017-04-25 14:12:50,223
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['hbase']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,224
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['hbase']
> > > > >
> > > > > 2017-04-25 14:12:50,271
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hcat']
> > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > > [u'hadoop']}
> > > > >
> > > > > 2017-04-25 14:12:50,272
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> user
> > > > > User['hcat']
> > > > >
> > > > > 2017-04-25 14:12:50,320
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > >
> > > > > 2017-04-25 14:12:50,333
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
> > > exist
> > > > >
> > > > > 2017-04-25 14:12:50,334
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to
> > 555
> > > > >
> > > > > 2017-04-25 14:12:50,335
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
> > > $(id
> > > > -u
> > > > > ambari-qa) -gt 1000) || (false)'}
> > > > >
> > > > > 2017-04-25 14:12:50,364
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > > > >
> > > > > 2017-04-25 14:12:50,365
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
> > > True,
> > > > > 'mode': 0775, 'cd_access': 'a'}
> > > > >
> > > > > 2017-04-25 14:12:50,366
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,366
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > owner
> > > > > for /tmp/hbase-hbase from 0 to hbase
> > > > >
> > > > > 2017-04-25 14:12:50,366
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > permission for /tmp/hbase-hbase from 755 to 775
> > > > >
> > > > > 2017-04-25 14:12:50,368
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > > >
> > > > > 2017-04-25 14:12:50,369
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > >
> > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> > >
> > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > > >
> > > > > 2017-04-25 14:12:50,393
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > > >
> > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> > >
> > > > > due to not_if
> > > > >
> > > > > 2017-04-25 14:12:50,394
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Group['hdfs']
> > > {}
> > > > >
> > > > > 2017-04-25 14:12:50,395
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
> group
> > > > > Group['hdfs']
> > > > >
> > > > > 2017-04-25 14:12:50,430
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hdfs']
> > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > > > >
> > > > > 2017-04-25 14:12:50,431
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
> > > user
> > > > > hdfs
> > > > >
> > > > > 2017-04-25 14:12:50,474
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > > > >
> > > > > 2017-04-25 14:12:50,474
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > > >
> > > > > 2017-04-25 14:12:50,474
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,475
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > > {'owner':
> > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > > >
> > > > > 2017-04-25 14:12:50,476
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
> > tmpdir']
> > >
> > > > > since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,476
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > owner
> > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
> > > > >
> > > > > 2017-04-25 14:12:50,476
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > group
> > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
> hadoop
> > > > >
> > > > > 2017-04-25 14:12:50,477
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
> > > 755
> > > > > to 1777
> > > > >
> > > > > 2017-04-25 14:12:50,477
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > > {'create_parents': True}
> > > > >
> > > > > 2017-04-25 14:12:50,478
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > since
> > > > > it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:50,479
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > > {'content':
> > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > > >
> > > > > 2017-04-25 14:12:50,504
> > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Downloading
> > > the
> > > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > > linux-x64.tar.gz
> > > > >
> > > > > 2017-04-25 14:12:53,793
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > > Directory['/usr/jdk64'] {}
> > > > >
> > > > > 2017-04-25 14:12:53,793
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > > >
> > > > > 2017-04-25 14:12:53,794
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > Execute[('chmod',
> > > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > > >
> > > > > 2017-04-25 14:12:53,824
> > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> Execute['cd
> > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> > ambari-sudo.sh
> > > cp
> > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > > > >
> > > > > 2017-04-25 14:13:00,439
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > > > ['delete']}
> > > > >
> > > > > 2017-04-25 14:13:00,440
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and
> > > all
> > > > > its content
> > > > >
> > > > > 2017-04-25 14:13:00,620
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access':
> > > 'a'}
> > > > >
> > > > > 2017-04-25 14:13:00,621
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Execute[('chmod',
> > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > > >
> > > > > 2017-04-25 14:13:00,682
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Initializing
> > > 2
> > > > > repositories
> > > > >
> > > > > 2017-04-25 14:13:00,683
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > > ['create'],
> > > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > > > 'mirror_list': None}
> > > > >
> > > > > 2017-04-25 14:13:00,709
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
> > > > >
> > > > > 2017-04-25 14:13:00,711
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > > >
> > > > > 2017-04-25 14:13:00,711
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': '
> http://public-repo-1.
>
> > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> > > ['create'],
> > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
> 'HDP-UTILS',
> > > > > 'mirror_list': None}
> > > > >
> > > > > 2017-04-25 14:13:00,717
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > > InlineTemplate(...)}
> > > > >
> > > > > 2017-04-25 14:13:00,717
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > > > >
> > > > > 2017-04-25 14:13:00,718
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Package['unzip']
> > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:00,846
> > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > > >
> > > > > 2017-04-25 14:13:03,413
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Package['curl']
> > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:03,429
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > > > installation of existing package curl
> > > > >
> > > > > 2017-04-25 14:13:03,430
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > > > 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:03,451
> > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
> > > > >
> > > > > 2017-04-25 14:13:04,948
> > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES
> > > Data
> > > > > Node
> > > > >
> > > > > 2017-04-25 14:13:04,952
> > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> > > False,
> > > > > 'retry_count': 5}
> > > > >
> > > > > 2017-04-25 14:13:05,086
> > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > > > elasticsearch-2.3.3')
> > > > >
> > > > >
> > > > >
> > > > > Command failed after 1 tries
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 13:47:17, Otto Fowler (
> ottobackwards@gmail.com)
>
> > > > > wrote:
> > > > >
> > > > > Yes, I’m going to run through again and get the logs. The basic
> issue
> > > > is,
> > > > > no repo tab -> if I should have one,
> > > > > and the ES yum command failing with nothing to do, which stops
> > > deployment
> > > > > on all the nodes.
> > > > >
> > > > >
> > > > >
> > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > > michael.miklavcic@gmail.com) wrote:
> > > > >
> > > > > I've been doing an internal hwx deployment (HCP) recently for
> testing
> > > > > purposes. I have some ES tweaks to work out, but other than that it
> > > has
> > > > > been running smoothly. Can you share any log info or screenshots?
> > > > >
> > > > > Mike
> > > > >
> > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Hi,
> > > > > > Should I see the repo tab in the ambari wizard for metron at this
> > > > point?
> > > > > > Asking for a friend who can’t get the wizard to work :)
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> >
>
--
Jon
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
I am following
https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html
I DO have master and data node together.
Why is that a problem?
I will try again with master and data node separate.
On April 26, 2017 at 10:41:59, Michael Miklavcic (
michael.miklavcic@gmail.com) wrote:
Hey Otto,
How do you have the ES nodes configured? For a base install I would setup 1
master (NOT as data node) and 2 data nodes (NOT on the same node as the
master). This is the install configuration I got working. You can also
modify some configuration properties around master node as data node, index
replicas, and gateway recovery to get it working differently, but this is
what will work OOTB with the default config settings from the mpack. If
you've already setup a master node and a data node on the same host, we'll
need to re-install.
Mike
On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <ot...@gmail.com>
wrote:
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
line
> 293, in _call
>
> raise ExecutionFailed(err_msg, code, out, err)
>
> ExecutionFailed: Execution of 'service elasticsearch status' returned 3.
●
> elasticsearch.service - Elasticsearch
>
> Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> disabled;
> vendor preset: disabled)
>
> Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58 EDT
> <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
>
> Docs: http://www.elastic.co
>
> Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch
> -Des.pidfile=${PID_DIR}/elasticsearch.pid -Des.default.path.home=${ES_
> HOME}
> -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR}
> -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
>
> Process: 16819
> ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec
> (code=exited, status=0/SUCCESS)
>
> Main PID: 16821 (code=exited, status=1/FAILURE)
>
>
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> org.elasticsearch.common.settings.Settings$Builder.
> loadFromStream(Settings.java:1080)
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> org.elasticsearch.common.settings.Settings$Builder.
> loadFromPath(Settings.java:1067)
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> org.elasticsearch.node.internal.InternalSettingsPreparer.
> prepareEnvironment(InternalSettingsPreparer.java:88)
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: Refer
> to
> the log for complete error details.
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> elasticsearch.service: main process exited, code=exited, status=1/FAILURE
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> elasticsearch.service entered failed state.
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> elasticsearch.service failed.
>
> INFO 2017-04-26 08:55:19,962
> <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
>
>
> On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> Can you describe what ES issues you are working on?
> Fixing the repos got everything installed, but my ES components don’t
stay
> running.
> I need to harvest the errors.
>
>
>
> On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> Nm. sorry. I fixed it.
>
>
>
> On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> Ok, now I see the repos in the ‘pick version’ screen, but it is erring on
> the f://localrepo
> even though the folder exists, there is no repodata/repomd.xml.
>
> What is the command to create a local repo?
>
>
>
> On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> I was going by the HW community page.
>
> Ok, Let me try it
>
>
>
> On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com) wrote:
>
> That would do it. It requires 2.4.2+. I would have sworn I put that in
the
> README, but I must have only annotated the PR. :(
>
> I'll get that in asap.
>
> -D...
>
>
> On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > 2.4.1 as a matter of fact
> >
> >
> > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com) wrote:
> >
> > Any chance they're running Ambari < 2.4.2?
> >
> > -D...
> >
> >
> > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > michael.miklavcic@gmail.com> wrote:
> >
> > > Hey Otto, I don't have wizard screenshots in front of me right now to
> > say
> > > for sure, but I do see a repoinfo.xml in the mpack. I haven't run
into
> > > anything like that, but next time I run through the install I can
take
> > > another look.
> > >
> > > https://github.com/apache/incubator-metron/blob/master/
> > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> > >
> > >
> > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ottobackwards@gmail.com
> >
> > > wrote:
> > >
> > > > stderr:
> > > >
> > > > Traceback (most recent call last):
> > > >
> > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > > >
> > > > Elasticsearch().execute()
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > libraries/script/script.py",
> > > > line 280, in execute
> > > >
> > > > method(env)
> > > >
> > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > > >
> > > > self.install_packages(env)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > libraries/script/script.py",
> > > > line 567, in install_packages
> > > >
> > > > retry_count=agent_stack_retry_count)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/base.py",
> > > > line 155, in __init__
> > > >
> > > > self.env.run()
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/environment.py",
> > > > line 160, in run
> > > >
> > > > self.run_action(resource, action)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/environment.py",
> > > > line 124, in run_action
> > > >
> > > > provider_action()
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/providers/package/__init__.py", line 54, in action_install
> > > >
> > > > self.install_package(package_name, self.resource.use_repos,
> > > > self.resource.skip_repos)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/providers/package/yumrpm.py", line 49, in install_package
> > > >
> > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > logoutput=self.get_logoutput())
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/providers/package/__init__.py", line 83, in
> > > checked_call_with_retries
> > > >
> > > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/providers/package/__init__.py", line 91, in _call_with_retries
> > > >
> > > > code, out = func(cmd, **kwargs)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/shell.py",
> > > > line 71, in inner
> > > >
> > > > result = function(command, **kwargs)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/shell.py",
> > > > line 93, in checked_call
> > > >
> > > > tries=tries, try_sleep=try_sleep)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/shell.py",
> > > > line 141, in _call_wrapper
> > > >
> > > > result = _call(command, **kwargs_copy)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/shell.py",
> > > > line 294, in _call
> > > >
> > > > raise Fail(err_msg)
> > > >
> > > > resource_management.core.exceptions.Fail: Execution of
'/usr/bin/yum
> > -d
> > > 0
> > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to
do
> > > >
> > > > stdout:
> > > >
> > > > 2017-04-25 14:12:48,669
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using
hadoop
> > > conf
> > > > dir: /usr/hdp/current/hadoop-client/conf
> > > >
> > > > 2017-04-25 14:12:48,671
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['metron']
> > > {}
> > > >
> > > > 2017-04-25 14:12:48,673
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
group
> > > > Group['metron']
> > > >
> > > > 2017-04-25 14:12:48,740
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['livy']
> > {}
> > > >
> > > > 2017-04-25 14:12:48,741
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
group
> > > > Group['livy']
> > > >
> > > > 2017-04-25 14:12:48,774
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['elasticsearch'] {}
> > > >
> > > > 2017-04-25 14:12:48,775
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
group
> > > > Group['elasticsearch']
> > > >
> > > > 2017-04-25 14:12:48,812
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['spark']
> > > {}
> > > >
> > > > 2017-04-25 14:12:48,812
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
group
> > > > Group['spark']
> > > >
> > > > 2017-04-25 14:12:48,848
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['zeppelin']
> > > > {}
> > > >
> > > > 2017-04-25 14:12:48,849
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
group
> > > > Group['zeppelin']
> > > >
> > > > 2017-04-25 14:12:48,884
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['hadoop']
> > > {}
> > > >
> > > > 2017-04-25 14:12:48,885
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
group
> > > > Group['hadoop']
> > > >
> > > > 2017-04-25 14:12:48,921
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['kibana']
> > > {}
> > > >
> > > > 2017-04-25 14:12:48,922
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding
group
> > > > Group['kibana']
> > > >
> > > > 2017-04-25 14:12:48,959
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['users']
> > > {}
> > > >
> > > > 2017-04-25 14:12:48,960
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
User['hive']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:48,961
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding user
> > > > User['hive']
> > > >
> > > > 2017-04-25 14:12:49,227
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['storm']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,227
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['storm']
> > > >
> > > > 2017-04-25 14:12:49,388
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['zookeeper']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,388
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['zookeeper']
> > > >
> > > > 2017-04-25 14:12:49,518
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > > > 'groups': [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,518
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['infra-solr']
> > > >
> > > > 2017-04-25 14:12:49,578
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ams']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,578
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['ams']
> > > >
> > > > 2017-04-25 14:12:49,625
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['tez']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'users']}
> > > >
> > > > 2017-04-25 14:12:49,626
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['tez']
> > > >
> > > > 2017-04-25 14:12:49,660
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['zeppelin']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,660
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['zeppelin']
> > > >
> > > > 2017-04-25 14:12:49,705
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['metron']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,705
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['metron']
> > > >
> > > > 2017-04-25 14:12:49,760
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
User['livy']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,760
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['livy']
> > > >
> > > > 2017-04-25 14:12:49,803
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> True,
> > > > 'groups': [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,804
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['elasticsearch']
> > > >
> > > > 2017-04-25 14:12:49,837
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['spark']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,837
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['spark']
> > > >
> > > > 2017-04-25 14:12:49,885
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['ambari-qa']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'users']}
> > > >
> > > > 2017-04-25 14:12:49,885
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['ambari-qa']
> > > >
> > > > 2017-04-25 14:12:49,928
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['flume']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,928
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['flume']
> > > >
> > > > 2017-04-25 14:12:49,972
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['kafka']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,973
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['kafka']
> > > >
> > > > 2017-04-25 14:12:50,025
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
User['hdfs']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,026
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['hdfs']
> > > >
> > > > 2017-04-25 14:12:50,075
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
User['yarn']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,075
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['yarn']
> > > >
> > > > 2017-04-25 14:12:50,125
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['kibana']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,126
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['kibana']
> > > >
> > > > 2017-04-25 14:12:50,175
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['mapred']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,176
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['mapred']
> > > >
> > > > 2017-04-25 14:12:50,223
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hbase']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,224
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['hbase']
> > > >
> > > > 2017-04-25 14:12:50,271
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
User['hcat']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,272
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['hcat']
> > > >
> > > > 2017-04-25 14:12:50,320
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > >
> > > > 2017-04-25 14:12:50,333
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
> > exist
> > > >
> > > > 2017-04-25 14:12:50,334
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to
> 555
> > > >
> > > > 2017-04-25 14:12:50,335
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
> > $(id
> > > -u
> > > > ambari-qa) -gt 1000) || (false)'}
> > > >
> > > > 2017-04-25 14:12:50,364
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > > >
> > > > 2017-04-25 14:12:50,365
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
> > True,
> > > > 'mode': 0775, 'cd_access': 'a'}
> > > >
> > > > 2017-04-25 14:12:50,366
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > > >
> > > > 2017-04-25 14:12:50,366
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > owner
> > > > for /tmp/hbase-hbase from 0 to hbase
> > > >
> > > > 2017-04-25 14:12:50,366
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > permission for /tmp/hbase-hbase from 755 to 775
> > > >
> > > > 2017-04-25 14:12:50,368
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > >
> > > > 2017-04-25 14:12:50,369
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > >
> /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> >
> > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > >
> > > > 2017-04-25 14:12:50,393
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > >
> /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> >
> > > > due to not_if
> > > >
> > > > 2017-04-25 14:12:50,394
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Group['hdfs']
> > {}
> > > >
> > > > 2017-04-25 14:12:50,395
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding
group
> > > > Group['hdfs']
> > > >
> > > > 2017-04-25 14:12:50,430
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
User['hdfs']
> > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > > >
> > > > 2017-04-25 14:12:50,431
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
> > user
> > > > hdfs
> > > >
> > > > 2017-04-25 14:12:50,474
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > > >
> > > > 2017-04-25 14:12:50,474
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > >
> > > > 2017-04-25 14:12:50,474
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > >
> > > > 2017-04-25 14:12:50,475
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > {'owner':
> > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > >
> > > > 2017-04-25 14:12:50,476
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
> tmpdir']
> >
> > > > since it doesn't exist.
> > > >
> > > > 2017-04-25 14:12:50,476
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > owner
> > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
> > > >
> > > > 2017-04-25 14:12:50,476
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > group
> > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to
hadoop
> > > >
> > > > 2017-04-25 14:12:50,477
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
> > 755
> > > > to 1777
> > > >
> > > > 2017-04-25 14:12:50,477
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > {'create_parents': True}
> > > >
> > > > 2017-04-25 14:12:50,478
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > since
> > > > it doesn't exist.
> > > >
> > > > 2017-04-25 14:12:50,479
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > {'content':
> > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > >
> > > > 2017-04-25 14:12:50,504
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Downloading
> > the
> > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > linux-x64.tar.gz
> > > >
> > > > 2017-04-25 14:12:53,793
> > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > Directory['/usr/jdk64'] {}
> > > >
> > > > 2017-04-25 14:12:53,793
> > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > >
> > > > 2017-04-25 14:12:53,794
> > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > Execute[('chmod',
> > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > >
> > > > 2017-04-25 14:12:53,824
> > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute['cd
> > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> ambari-sudo.sh
> > cp
> > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > > >
> > > > 2017-04-25 14:13:00,439
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > > ['delete']}
> > > >
> > > > 2017-04-25 14:13:00,440
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and
> > all
> > > > its content
> > > >
> > > > 2017-04-25 14:13:00,620
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access':
> > 'a'}
> > > >
> > > > 2017-04-25 14:13:00,621
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Execute[('chmod',
> > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > >
> > > > 2017-04-25 14:13:00,682
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
Initializing
> > 2
> > > > repositories
> > > >
> > > > 2017-04-25 14:13:00,683
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > ['create'],
> > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > > 'mirror_list': None}
> > > >
> > > > 2017-04-25 14:13:00,709
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
> > > >
> > > > 2017-04-25 14:13:00,711
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > >
> > > > 2017-04-25 14:13:00,711
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://public-repo-1.
> > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> > ['create'],
> > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name':
'HDP-UTILS',
> > > > 'mirror_list': None}
> > > >
> > > > 2017-04-25 14:13:00,717
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > InlineTemplate(...)}
> > > >
> > > > 2017-04-25 14:13:00,717
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > > >
> > > > 2017-04-25 14:13:00,718
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Package['unzip']
> > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > >
> > > > 2017-04-25 14:13:00,846
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > >
> > > > 2017-04-25 14:13:03,413
> > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > Package['curl']
> > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > >
> > > > 2017-04-25 14:13:03,429
> > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > > installation of existing package curl
> > > >
> > > > 2017-04-25 14:13:03,430
> > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > > 'retry_count': 5}
> > > >
> > > > 2017-04-25 14:13:03,451
> > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
> > > >
> > > > 2017-04-25 14:13:04,948
> > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES
> > Data
> > > > Node
> > > >
> > > > 2017-04-25 14:13:04,952
> > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> > False,
> > > > 'retry_count': 5}
> > > >
> > > > 2017-04-25 14:13:05,086
> > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > > elasticsearch-2.3.3')
> > > >
> > > >
> > > >
> > > > Command failed after 1 tries
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 13:47:17, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > Yes, I’m going to run through again and get the logs. The basic
issue
> > > is,
> > > > no repo tab -> if I should have one,
> > > > and the ES yum command failing with nothing to do, which stops
> > deployment
> > > > on all the nodes.
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > michael.miklavcic@gmail.com) wrote:
> > > >
> > > > I've been doing an internal hwx deployment (HCP) recently for
testing
> > > > purposes. I have some ES tweaks to work out, but other than that it
> > has
> > > > been running smoothly. Can you share any log info or screenshots?
> > > >
> > > > Mike
> > > >
> > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
> > wrote:
> > > >
> > > > > Hi,
> > > > > Should I see the repo tab in the ambari wizard for metron at this
> > > point?
> > > > > Asking for a friend who can’t get the wizard to work :)
> > > > >
> > > >
> > > >
> > >
> >
> >
>
Re: Ambari Wizard: Repo Tab
Posted by Michael Miklavcic <mi...@gmail.com>.
Hey Otto,
How do you have the ES nodes configured? For a base install I would setup 1
master (NOT as data node) and 2 data nodes (NOT on the same node as the
master). This is the install configuration I got working. You can also
modify some configuration properties around master node as data node, index
replicas, and gateway recovery to get it working differently, but this is
what will work OOTB with the default config settings from the mpack. If
you've already setup a master node and a data node on the same host, we'll
need to re-install.
Mike
On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <ot...@gmail.com>
wrote:
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line
> 293, in _call
>
> raise ExecutionFailed(err_msg, code, out, err)
>
> ExecutionFailed: Execution of 'service elasticsearch status' returned 3. ●
> elasticsearch.service - Elasticsearch
>
> Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service;
> disabled;
> vendor preset: disabled)
>
> Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58 EDT
> <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
>
> Docs: http://www.elastic.co
>
> Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch
> -Des.pidfile=${PID_DIR}/elasticsearch.pid -Des.default.path.home=${ES_
> HOME}
> -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR}
> -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
>
> Process: 16819
> ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec
> (code=exited, status=0/SUCCESS)
>
> Main PID: 16821 (code=exited, status=1/FAILURE)
>
>
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> org.elasticsearch.common.settings.Settings$Builder.
> loadFromStream(Settings.java:1080)
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> org.elasticsearch.common.settings.Settings$Builder.
> loadFromPath(Settings.java:1067)
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> org.elasticsearch.node.internal.InternalSettingsPreparer.
> prepareEnvironment(InternalSettingsPreparer.java:88)
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
> org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: Refer
> to
> the log for complete error details.
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> elasticsearch.service: main process exited, code=exited, status=1/FAILURE
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
> elasticsearch.service entered failed state.
>
> Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
> ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
> elasticsearch.service failed.
>
> INFO 2017-04-26 08:55:19,962
> <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
>
>
> On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> Can you describe what ES issues you are working on?
> Fixing the repos got everything installed, but my ES components don’t stay
> running.
> I need to harvest the errors.
>
>
>
> On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> Nm. sorry. I fixed it.
>
>
>
> On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> Ok, now I see the repos in the ‘pick version’ screen, but it is erring on
> the f://localrepo
> even though the folder exists, there is no repodata/repomd.xml.
>
> What is the command to create a local repo?
>
>
>
> On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> I was going by the HW community page.
>
> Ok, Let me try it
>
>
>
> On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com) wrote:
>
> That would do it. It requires 2.4.2+. I would have sworn I put that in the
> README, but I must have only annotated the PR. :(
>
> I'll get that in asap.
>
> -D...
>
>
> On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > 2.4.1 as a matter of fact
> >
> >
> > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com) wrote:
> >
> > Any chance they're running Ambari < 2.4.2?
> >
> > -D...
> >
> >
> > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > michael.miklavcic@gmail.com> wrote:
> >
> > > Hey Otto, I don't have wizard screenshots in front of me right now to
> > say
> > > for sure, but I do see a repoinfo.xml in the mpack. I haven't run into
> > > anything like that, but next time I run through the install I can take
> > > another look.
> > >
> > > https://github.com/apache/incubator-metron/blob/master/
> > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> > >
> > >
> > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ottobackwards@gmail.com
> >
> > > wrote:
> > >
> > > > stderr:
> > > >
> > > > Traceback (most recent call last):
> > > >
> > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > > >
> > > > Elasticsearch().execute()
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > libraries/script/script.py",
> > > > line 280, in execute
> > > >
> > > > method(env)
> > > >
> > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > > >
> > > > self.install_packages(env)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > libraries/script/script.py",
> > > > line 567, in install_packages
> > > >
> > > > retry_count=agent_stack_retry_count)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/base.py",
> > > > line 155, in __init__
> > > >
> > > > self.env.run()
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/environment.py",
> > > > line 160, in run
> > > >
> > > > self.run_action(resource, action)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/environment.py",
> > > > line 124, in run_action
> > > >
> > > > provider_action()
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/providers/package/__init__.py", line 54, in action_install
> > > >
> > > > self.install_package(package_name, self.resource.use_repos,
> > > > self.resource.skip_repos)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/providers/package/yumrpm.py", line 49, in install_package
> > > >
> > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > logoutput=self.get_logoutput())
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/providers/package/__init__.py", line 83, in
> > > checked_call_with_retries
> > > >
> > > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/providers/package/__init__.py", line 91, in _call_with_retries
> > > >
> > > > code, out = func(cmd, **kwargs)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/shell.py",
> > > > line 71, in inner
> > > >
> > > > result = function(command, **kwargs)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/shell.py",
> > > > line 93, in checked_call
> > > >
> > > > tries=tries, try_sleep=try_sleep)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/shell.py",
> > > > line 141, in _call_wrapper
> > > >
> > > > result = _call(command, **kwargs_copy)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/shell.py",
> > > > line 294, in _call
> > > >
> > > > raise Fail(err_msg)
> > > >
> > > > resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum
> > -d
> > > 0
> > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to do
> > > >
> > > > stdout:
> > > >
> > > > 2017-04-25 14:12:48,669
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using hadoop
> > > conf
> > > > dir: /usr/hdp/current/hadoop-client/conf
> > > >
> > > > 2017-04-25 14:12:48,671
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['metron']
> > > {}
> > > >
> > > > 2017-04-25 14:12:48,673
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > > Group['metron']
> > > >
> > > > 2017-04-25 14:12:48,740
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['livy']
> > {}
> > > >
> > > > 2017-04-25 14:12:48,741
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > > Group['livy']
> > > >
> > > > 2017-04-25 14:12:48,774
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['elasticsearch'] {}
> > > >
> > > > 2017-04-25 14:12:48,775
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > > Group['elasticsearch']
> > > >
> > > > 2017-04-25 14:12:48,812
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['spark']
> > > {}
> > > >
> > > > 2017-04-25 14:12:48,812
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > > Group['spark']
> > > >
> > > > 2017-04-25 14:12:48,848
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['zeppelin']
> > > > {}
> > > >
> > > > 2017-04-25 14:12:48,849
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > > Group['zeppelin']
> > > >
> > > > 2017-04-25 14:12:48,884
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['hadoop']
> > > {}
> > > >
> > > > 2017-04-25 14:12:48,885
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > > Group['hadoop']
> > > >
> > > > 2017-04-25 14:12:48,921
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['kibana']
> > > {}
> > > >
> > > > 2017-04-25 14:12:48,922
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > > Group['kibana']
> > > >
> > > > 2017-04-25 14:12:48,959
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['users']
> > > {}
> > > >
> > > > 2017-04-25 14:12:48,960
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - User['hive']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:48,961
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding user
> > > > User['hive']
> > > >
> > > > 2017-04-25 14:12:49,227
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['storm']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,227
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['storm']
> > > >
> > > > 2017-04-25 14:12:49,388
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['zookeeper']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,388
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['zookeeper']
> > > >
> > > > 2017-04-25 14:12:49,518
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > > > 'groups': [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,518
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['infra-solr']
> > > >
> > > > 2017-04-25 14:12:49,578
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ams']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,578
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['ams']
> > > >
> > > > 2017-04-25 14:12:49,625
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['tez']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'users']}
> > > >
> > > > 2017-04-25 14:12:49,626
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['tez']
> > > >
> > > > 2017-04-25 14:12:49,660
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['zeppelin']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,660
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['zeppelin']
> > > >
> > > > 2017-04-25 14:12:49,705
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['metron']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,705
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['metron']
> > > >
> > > > 2017-04-25 14:12:49,760
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['livy']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,760
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['livy']
> > > >
> > > > 2017-04-25 14:12:49,803
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> True,
> > > > 'groups': [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,804
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['elasticsearch']
> > > >
> > > > 2017-04-25 14:12:49,837
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['spark']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,837
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['spark']
> > > >
> > > > 2017-04-25 14:12:49,885
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['ambari-qa']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'users']}
> > > >
> > > > 2017-04-25 14:12:49,885
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['ambari-qa']
> > > >
> > > > 2017-04-25 14:12:49,928
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['flume']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,928
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['flume']
> > > >
> > > > 2017-04-25 14:12:49,972
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['kafka']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,973
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['kafka']
> > > >
> > > > 2017-04-25 14:12:50,025
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,026
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['hdfs']
> > > >
> > > > 2017-04-25 14:12:50,075
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['yarn']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,075
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['yarn']
> > > >
> > > > 2017-04-25 14:12:50,125
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['kibana']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,126
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['kibana']
> > > >
> > > > 2017-04-25 14:12:50,175
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['mapred']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,176
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['mapred']
> > > >
> > > > 2017-04-25 14:12:50,223
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hbase']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,224
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['hbase']
> > > >
> > > > 2017-04-25 14:12:50,271
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hcat']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,272
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['hcat']
> > > >
> > > > 2017-04-25 14:12:50,320
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > >
> > > > 2017-04-25 14:12:50,333
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
> > exist
> > > >
> > > > 2017-04-25 14:12:50,334
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to
> 555
> > > >
> > > > 2017-04-25 14:12:50,335
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
> > $(id
> > > -u
> > > > ambari-qa) -gt 1000) || (false)'}
> > > >
> > > > 2017-04-25 14:12:50,364
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > > >
> > > > 2017-04-25 14:12:50,365
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
> > True,
> > > > 'mode': 0775, 'cd_access': 'a'}
> > > >
> > > > 2017-04-25 14:12:50,366
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > > >
> > > > 2017-04-25 14:12:50,366
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > owner
> > > > for /tmp/hbase-hbase from 0 to hbase
> > > >
> > > > 2017-04-25 14:12:50,366
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > permission for /tmp/hbase-hbase from 755 to 775
> > > >
> > > > 2017-04-25 14:12:50,368
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > >
> > > > 2017-04-25 14:12:50,369
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > >
> /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> >
> > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > >
> > > > 2017-04-25 14:12:50,393
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > >
> /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> >
> > > > due to not_if
> > > >
> > > > 2017-04-25 14:12:50,394
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Group['hdfs']
> > {}
> > > >
> > > > 2017-04-25 14:12:50,395
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding group
> > > > Group['hdfs']
> > > >
> > > > 2017-04-25 14:12:50,430
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > > >
> > > > 2017-04-25 14:12:50,431
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
> > user
> > > > hdfs
> > > >
> > > > 2017-04-25 14:12:50,474
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > > >
> > > > 2017-04-25 14:12:50,474
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > >
> > > > 2017-04-25 14:12:50,474
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > >
> > > > 2017-04-25 14:12:50,475
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > {'owner':
> > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > >
> > > > 2017-04-25 14:12:50,476
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
> tmpdir']
> >
> > > > since it doesn't exist.
> > > >
> > > > 2017-04-25 14:12:50,476
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > owner
> > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
> > > >
> > > > 2017-04-25 14:12:50,476
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > group
> > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hadoop
> > > >
> > > > 2017-04-25 14:12:50,477
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
> > 755
> > > > to 1777
> > > >
> > > > 2017-04-25 14:12:50,477
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > {'create_parents': True}
> > > >
> > > > 2017-04-25 14:12:50,478
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > since
> > > > it doesn't exist.
> > > >
> > > > 2017-04-25 14:12:50,479
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > {'content':
> > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > >
> > > > 2017-04-25 14:12:50,504
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Downloading
> > the
> > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > linux-x64.tar.gz
> > > >
> > > > 2017-04-25 14:12:53,793
> > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > Directory['/usr/jdk64'] {}
> > > >
> > > > 2017-04-25 14:12:53,793
> > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > >
> > > > 2017-04-25 14:12:53,794
> > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > Execute[('chmod',
> > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > >
> > > > 2017-04-25 14:12:53,824
> > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute['cd
> > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> ambari-sudo.sh
> > cp
> > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > > >
> > > > 2017-04-25 14:13:00,439
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > > ['delete']}
> > > >
> > > > 2017-04-25 14:13:00,440
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and
> > all
> > > > its content
> > > >
> > > > 2017-04-25 14:13:00,620
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access':
> > 'a'}
> > > >
> > > > 2017-04-25 14:13:00,621
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Execute[('chmod',
> > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > >
> > > > 2017-04-25 14:13:00,682
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Initializing
> > 2
> > > > repositories
> > > >
> > > > 2017-04-25 14:13:00,683
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > ['create'],
> > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > > 'mirror_list': None}
> > > >
> > > > 2017-04-25 14:13:00,709
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
> > > >
> > > > 2017-04-25 14:13:00,711
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > >
> > > > 2017-04-25 14:13:00,711
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://public-repo-1.
> > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> > ['create'],
> > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS',
> > > > 'mirror_list': None}
> > > >
> > > > 2017-04-25 14:13:00,717
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > InlineTemplate(...)}
> > > >
> > > > 2017-04-25 14:13:00,717
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > > >
> > > > 2017-04-25 14:13:00,718
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Package['unzip']
> > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > >
> > > > 2017-04-25 14:13:00,846
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > >
> > > > 2017-04-25 14:13:03,413
> > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > Package['curl']
> > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > >
> > > > 2017-04-25 14:13:03,429
> > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > > installation of existing package curl
> > > >
> > > > 2017-04-25 14:13:03,430
> > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > > 'retry_count': 5}
> > > >
> > > > 2017-04-25 14:13:03,451
> > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
> > > >
> > > > 2017-04-25 14:13:04,948
> > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES
> > Data
> > > > Node
> > > >
> > > > 2017-04-25 14:13:04,952
> > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> > False,
> > > > 'retry_count': 5}
> > > >
> > > > 2017-04-25 14:13:05,086
> > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > > elasticsearch-2.3.3')
> > > >
> > > >
> > > >
> > > > Command failed after 1 tries
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 13:47:17, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > Yes, I’m going to run through again and get the logs. The basic issue
> > > is,
> > > > no repo tab -> if I should have one,
> > > > and the ES yum command failing with nothing to do, which stops
> > deployment
> > > > on all the nodes.
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > michael.miklavcic@gmail.com) wrote:
> > > >
> > > > I've been doing an internal hwx deployment (HCP) recently for testing
> > > > purposes. I have some ES tweaks to work out, but other than that it
> > has
> > > > been running smoothly. Can you share any log info or screenshots?
> > > >
> > > > Mike
> > > >
> > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
> > wrote:
> > > >
> > > > > Hi,
> > > > > Should I see the repo tab in the ambari wizard for metron at this
> > > point?
> > > > > Asking for a friend who can’t get the wizard to work :)
> > > > >
> > > >
> > > >
> > >
> >
> >
>
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
File
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line
293, in _call
raise ExecutionFailed(err_msg, code, out, err)
ExecutionFailed: Execution of 'service elasticsearch status' returned 3. ●
elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled;
vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58 EDT
<http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago
Docs: http://www.elastic.co
Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch
-Des.pidfile=${PID_DIR}/elasticsearch.pid -Des.default.path.home=${ES_HOME}
-Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR}
-Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
Process: 16819
ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec
(code=exited, status=0/SUCCESS)
Main PID: 16821 (code=exited, status=1/FAILURE)
Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:1080)
Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1067)
Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:88)
Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at
org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: Refer to
the log for complete error details.
Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit
elasticsearch.service entered failed state.
Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>
ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]:
elasticsearch.service failed.
INFO 2017-04-26 08:55:19,962
<http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr
On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwards@gmail.com) wrote:
Can you describe what ES issues you are working on?
Fixing the repos got everything installed, but my ES components don’t stay
running.
I need to harvest the errors.
On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com) wrote:
Nm. sorry. I fixed it.
On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com) wrote:
Ok, now I see the repos in the ‘pick version’ screen, but it is erring on
the f://localrepo
even though the folder exists, there is no repodata/repomd.xml.
What is the command to create a local repo?
On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com) wrote:
I was going by the HW community page.
Ok, Let me try it
On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com) wrote:
That would do it. It requires 2.4.2+. I would have sworn I put that in the
README, but I must have only annotated the PR. :(
I'll get that in asap.
-D...
On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
wrote:
> 2.4.1 as a matter of fact
>
>
> On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com) wrote:
>
> Any chance they're running Ambari < 2.4.2?
>
> -D...
>
>
> On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> michael.miklavcic@gmail.com> wrote:
>
> > Hey Otto, I don't have wizard screenshots in front of me right now to
> say
> > for sure, but I do see a repoinfo.xml in the mpack. I haven't run into
> > anything like that, but next time I run through the install I can take
> > another look.
> >
> > https://github.com/apache/incubator-metron/blob/master/
> > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> >
> >
> > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > stderr:
> > >
> > > Traceback (most recent call last):
> > >
> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > >
> > > Elasticsearch().execute()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > libraries/script/script.py",
> > > line 280, in execute
> > >
> > > method(env)
> > >
> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > >
> > > self.install_packages(env)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > libraries/script/script.py",
> > > line 567, in install_packages
> > >
> > > retry_count=agent_stack_retry_count)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/base.py",
> > > line 155, in __init__
> > >
> > > self.env.run()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/environment.py",
> > > line 160, in run
> > >
> > > self.run_action(resource, action)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/environment.py",
> > > line 124, in run_action
> > >
> > > provider_action()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 54, in action_install
> > >
> > > self.install_package(package_name, self.resource.use_repos,
> > > self.resource.skip_repos)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/yumrpm.py", line 49, in install_package
> > >
> > > self.checked_call_with_retries(cmd, sudo=True,
> > > logoutput=self.get_logoutput())
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 83, in
> > checked_call_with_retries
> > >
> > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 91, in _call_with_retries
> > >
> > > code, out = func(cmd, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 71, in inner
> > >
> > > result = function(command, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 93, in checked_call
> > >
> > > tries=tries, try_sleep=try_sleep)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 141, in _call_wrapper
> > >
> > > result = _call(command, **kwargs_copy)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 294, in _call
> > >
> > > raise Fail(err_msg)
> > >
> > > resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum
> -d
> > 0
> > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to do
> > >
> > > stdout:
> > >
> > > 2017-04-25 14:12:48,669
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using hadoop
> > conf
> > > dir: /usr/hdp/current/hadoop-client/conf
> > >
> > > 2017-04-25 14:12:48,671
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['metron']
> > {}
> > >
> > > 2017-04-25 14:12:48,673
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['metron']
> > >
> > > 2017-04-25 14:12:48,740
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['livy']
> {}
> > >
> > > 2017-04-25 14:12:48,741
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['livy']
> > >
> > > 2017-04-25 14:12:48,774
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['elasticsearch'] {}
> > >
> > > 2017-04-25 14:12:48,775
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['elasticsearch']
> > >
> > > 2017-04-25 14:12:48,812
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['spark']
> > {}
> > >
> > > 2017-04-25 14:12:48,812
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['spark']
> > >
> > > 2017-04-25 14:12:48,848
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['zeppelin']
> > > {}
> > >
> > > 2017-04-25 14:12:48,849
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['zeppelin']
> > >
> > > 2017-04-25 14:12:48,884
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['hadoop']
> > {}
> > >
> > > 2017-04-25 14:12:48,885
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['hadoop']
> > >
> > > 2017-04-25 14:12:48,921
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['kibana']
> > {}
> > >
> > > 2017-04-25 14:12:48,922
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['kibana']
> > >
> > > 2017-04-25 14:12:48,959
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['users']
> > {}
> > >
> > > 2017-04-25 14:12:48,960
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - User['hive']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:48,961
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding user
> > > User['hive']
> > >
> > > 2017-04-25 14:12:49,227
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['storm']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,227
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['storm']
> > >
> > > 2017-04-25 14:12:49,388
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['zookeeper']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,388
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['zookeeper']
> > >
> > > 2017-04-25 14:12:49,518
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > > 'groups': [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,518
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['infra-solr']
> > >
> > > 2017-04-25 14:12:49,578
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ams']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,578
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['ams']
> > >
> > > 2017-04-25 14:12:49,625
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['tez']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> > >
> > > 2017-04-25 14:12:49,626
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['tez']
> > >
> > > 2017-04-25 14:12:49,660
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['zeppelin']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,660
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['zeppelin']
> > >
> > > 2017-04-25 14:12:49,705
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['metron']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,705
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['metron']
> > >
> > > 2017-04-25 14:12:49,760
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['livy']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,760
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['livy']
> > >
> > > 2017-04-25 14:12:49,803
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > > 'groups': [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,804
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['elasticsearch']
> > >
> > > 2017-04-25 14:12:49,837
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['spark']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,837
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['spark']
> > >
> > > 2017-04-25 14:12:49,885
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['ambari-qa']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> > >
> > > 2017-04-25 14:12:49,885
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['ambari-qa']
> > >
> > > 2017-04-25 14:12:49,928
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['flume']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,928
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['flume']
> > >
> > > 2017-04-25 14:12:49,972
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['kafka']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,973
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['kafka']
> > >
> > > 2017-04-25 14:12:50,025
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,026
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hdfs']
> > >
> > > 2017-04-25 14:12:50,075
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['yarn']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,075
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['yarn']
> > >
> > > 2017-04-25 14:12:50,125
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['kibana']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,126
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['kibana']
> > >
> > > 2017-04-25 14:12:50,175
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['mapred']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,176
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['mapred']
> > >
> > > 2017-04-25 14:12:50,223
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hbase']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,224
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hbase']
> > >
> > > 2017-04-25 14:12:50,271
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hcat']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,272
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hcat']
> > >
> > > 2017-04-25 14:12:50,320
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > >
> > > 2017-04-25 14:12:50,333
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
> exist
> > >
> > > 2017-04-25 14:12:50,334
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to 555
> > >
> > > 2017-04-25 14:12:50,335
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
> $(id
> > -u
> > > ambari-qa) -gt 1000) || (false)'}
> > >
> > > 2017-04-25 14:12:50,364
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > >
> > > 2017-04-25 14:12:50,365
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
> True,
> > > 'mode': 0775, 'cd_access': 'a'}
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> owner
> > > for /tmp/hbase-hbase from 0 to hbase
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /tmp/hbase-hbase from 755 to 775
> > >
> > > 2017-04-25 14:12:50,368
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > >
> > > 2017-04-25 14:12:50,369
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > >
/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>
> > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > >
> > > 2017-04-25 14:12:50,393
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > >
/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>
> > > due to not_if
> > >
> > > 2017-04-25 14:12:50,394
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Group['hdfs']
> {}
> > >
> > > 2017-04-25 14:12:50,395
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding group
> > > Group['hdfs']
> > >
> > > 2017-04-25 14:12:50,430
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > >
> > > 2017-04-25 14:12:50,431
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
> user
> > > hdfs
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/etc/hadoop'] {'mode': 0755}
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,475
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> {'owner':
> > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
>
> > > since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> owner
> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> group
> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hadoop
> > >
> > > 2017-04-25 14:12:50,477
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
> 755
> > > to 1777
> > >
> > > 2017-04-25 14:12:50,477
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > {'create_parents': True}
> > >
> > > 2017-04-25 14:12:50,478
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> since
> > > it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,479
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> {'content':
> > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > >
> > > 2017-04-25 14:12:50,504
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Downloading
> the
> > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > linux-x64.tar.gz
> > >
> > > 2017-04-25 14:12:53,793
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > Directory['/usr/jdk64'] {}
> > >
> > > 2017-04-25 14:12:53,793
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:53,794
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > Execute[('chmod',
> > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > >
> > > 2017-04-25 14:12:53,824
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute['cd
> > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz && ambari-sudo.sh
> cp
> > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > >
> > > 2017-04-25 14:13:00,439
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > ['delete']}
> > >
> > > 2017-04-25 14:13:00,440
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and
> all
> > > its content
> > >
> > > 2017-04-25 14:13:00,620
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access':
> 'a'}
> > >
> > > 2017-04-25 14:13:00,621
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Execute[('chmod',
> > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > >
> > > 2017-04-25 14:13:00,682
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Initializing
> 2
> > > repositories
> > >
> > > 2017-04-25 14:13:00,683
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> ['create'],
> > > 'components': [u'HDP', 'main'], 'repo_template':
> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > 'mirror_list': None}
> > >
> > > 2017-04-25 14:13:00,709
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
> > >
> > > 2017-04-25 14:13:00,711
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > >
> > > 2017-04-25 14:13:00,711
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://public-repo-1.
> > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> ['create'],
> > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS',
> > > 'mirror_list': None}
> > >
> > > 2017-04-25 14:13:00,717
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> InlineTemplate(...)}
> > >
> > > 2017-04-25 14:13:00,717
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > >
> > > 2017-04-25 14:13:00,718
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Package['unzip']
> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:00,846
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > >
> > > 2017-04-25 14:13:03,413
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> Package['curl']
> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:03,429
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > installation of existing package curl
> > >
> > > 2017-04-25 14:13:03,430
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:03,451
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
> > >
> > > 2017-04-25 14:13:04,948
> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES
> Data
> > > Node
> > >
> > > 2017-04-25 14:13:04,952
> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> False,
> > > 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:05,086
> > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > elasticsearch-2.3.3')
> > >
> > >
> > >
> > > Command failed after 1 tries
> > >
> > >
> > >
> > >
> > >
> > > On April 25, 2017 at 13:47:17, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > Yes, I’m going to run through again and get the logs. The basic issue
> > is,
> > > no repo tab -> if I should have one,
> > > and the ES yum command failing with nothing to do, which stops
> deployment
> > > on all the nodes.
> > >
> > >
> > >
> > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > michael.miklavcic@gmail.com) wrote:
> > >
> > > I've been doing an internal hwx deployment (HCP) recently for testing
> > > purposes. I have some ES tweaks to work out, but other than that it
> has
> > > been running smoothly. Can you share any log info or screenshots?
> > >
> > > Mike
> > >
> > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > > Should I see the repo tab in the ambari wizard for metron at this
> > point?
> > > > Asking for a friend who can’t get the wizard to work :)
> > > >
> > >
> > >
> >
>
>
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
Updating to use ambari 2.4.2 fixed it for me.
You will see the repos in the beginning of the wizard, mixed in with the
HDP repos now.
The steps in the community page where you ‘just’ copy the rpm’s to the
/localrepo don’t work anymore
though, because you have to run create repo there.
On April 26, 2017 at 08:58:30, Anand Subramanian (
asubramanian@hortonworks.com) wrote:
FWIW, I don’t see the Repository tab as well in recent builds.
Here's a screenshot of how it was earlier:
https://community.hortonworks.com/storage/attachments/9844-metron-screen2.png
And here's one from a recent build:
http://imgur.com/yjkltES
Regards,
Anand
On 4/26/17, 5:57 PM, "Otto Fowler" <ot...@gmail.com> wrote:
>Can you describe what ES issues you are working on?
>Fixing the repos got everything installed, but my ES components don’t stay
>running.
>I need to harvest the errors.
>
>
>
>On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com)
wrote:
>
>Nm. sorry. I fixed it.
>
>
>
>On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com)
wrote:
>
>Ok, now I see the repos in the ‘pick version’ screen, but it is erring on
>the f://localrepo
>even though the folder exists, there is no repodata/repomd.xml.
>
>What is the command to create a local repo?
>
>
>
>On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com)
wrote:
>
>I was going by the HW community page.
>
>Ok, Let me try it
>
>
>
>On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com) wrote:
>
>That would do it. It requires 2.4.2+. I would have sworn I put that in the
>README, but I must have only annotated the PR. :(
>
>I'll get that in asap.
>
>-D...
>
>
>On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
>wrote:
>
>> 2.4.1 as a matter of fact
>>
>>
>> On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com) wrote:
>>
>> Any chance they're running Ambari < 2.4.2?
>>
>> -D...
>>
>>
>> On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
>> michael.miklavcic@gmail.com> wrote:
>>
>> > Hey Otto, I don't have wizard screenshots in front of me right now to
>> say
>> > for sure, but I do see a repoinfo.xml in the mpack. I haven't run into
>> > anything like that, but next time I run through the install I can take
>> > another look.
>> >
>> > https://github.com/apache/incubator-metron/blob/master/
>> > metron-deployment/packaging/ambari/metron-mpack/src/main/
>> > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
>> >
>> >
>> > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ot...@gmail.com>
>> > wrote:
>> >
>> > > stderr:
>> > >
>> > > Traceback (most recent call last):
>> > >
>> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
>> > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
>> > >
>> > > Elasticsearch().execute()
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > libraries/script/script.py",
>> > > line 280, in execute
>> > >
>> > > method(env)
>> > >
>> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
>> > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
>> > >
>> > > self.install_packages(env)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > libraries/script/script.py",
>> > > line 567, in install_packages
>> > >
>> > > retry_count=agent_stack_retry_count)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > core/base.py",
>> > > line 155, in __init__
>> > >
>> > > self.env.run()
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > core/environment.py",
>> > > line 160, in run
>> > >
>> > > self.run_action(resource, action)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > core/environment.py",
>> > > line 124, in run_action
>> > >
>> > > provider_action()
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > core/providers/package/__init__.py", line 54, in action_install
>> > >
>> > > self.install_package(package_name, self.resource.use_repos,
>> > > self.resource.skip_repos)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > core/providers/package/yumrpm.py", line 49, in install_package
>> > >
>> > > self.checked_call_with_retries(cmd, sudo=True,
>> > > logoutput=self.get_logoutput())
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > core/providers/package/__init__.py", line 83, in
>> > checked_call_with_retries
>> > >
>> > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > core/providers/package/__init__.py", line 91, in _call_with_retries
>> > >
>> > > code, out = func(cmd, **kwargs)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > core/shell.py",
>> > > line 71, in inner
>> > >
>> > > result = function(command, **kwargs)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > core/shell.py",
>> > > line 93, in checked_call
>> > >
>> > > tries=tries, try_sleep=try_sleep)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > core/shell.py",
>> > > line 141, in _call_wrapper
>> > >
>> > > result = _call(command, **kwargs_copy)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > core/shell.py",
>> > > line 294, in _call
>> > >
>> > > raise Fail(err_msg)
>> > >
>> > > resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum
>> -d
>> > 0
>> > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to
do
>> > >
>> > > stdout:
>> > >
>> > > 2017-04-25 14:12:48,669
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using hadoop
>> > conf
>> > > dir: /usr/hdp/current/hadoop-client/conf
>> > >
>> > > 2017-04-25 14:12:48,671
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Group['metron']
>> > {}
>> > >
>> > > 2017-04-25 14:12:48,673
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
>> > > Group['metron']
>> > >
>> > > 2017-04-25 14:12:48,740
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Group['livy']
>> {}
>> > >
>> > > 2017-04-25 14:12:48,741
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
>> > > Group['livy']
>> > >
>> > > 2017-04-25 14:12:48,774
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> > > Group['elasticsearch'] {}
>> > >
>> > > 2017-04-25 14:12:48,775
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
>> > > Group['elasticsearch']
>> > >
>> > > 2017-04-25 14:12:48,812
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Group['spark']
>> > {}
>> > >
>> > > 2017-04-25 14:12:48,812
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
>> > > Group['spark']
>> > >
>> > > 2017-04-25 14:12:48,848
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> > Group['zeppelin']
>> > > {}
>> > >
>> > > 2017-04-25 14:12:48,849
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
>> > > Group['zeppelin']
>> > >
>> > > 2017-04-25 14:12:48,884
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Group['hadoop']
>> > {}
>> > >
>> > > 2017-04-25 14:12:48,885
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
>> > > Group['hadoop']
>> > >
>> > > 2017-04-25 14:12:48,921
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Group['kibana']
>> > {}
>> > >
>> > > 2017-04-25 14:12:48,922
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
>> > > Group['kibana']
>> > >
>> > > 2017-04-25 14:12:48,959
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Group['users']
>> > {}
>> > >
>> > > 2017-04-25 14:12:48,960
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - User['hive']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:48,961
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding user
>> > > User['hive']
>> > >
>> > > 2017-04-25 14:12:49,227
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
User['storm']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,227
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['storm']
>> > >
>> > > 2017-04-25 14:12:49,388
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > User['zookeeper']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,388
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['zookeeper']
>> > >
>> > > 2017-04-25 14:12:49,518
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
>> > > 'groups': [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,518
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['infra-solr']
>> > >
>> > > 2017-04-25 14:12:49,578
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ams']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,578
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['ams']
>> > >
>> > > 2017-04-25 14:12:49,625
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['tez']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
[u'users']}
>> > >
>> > > 2017-04-25 14:12:49,626
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['tez']
>> > >
>> > > 2017-04-25 14:12:49,660
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> User['zeppelin']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,660
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['zeppelin']
>> > >
>> > > 2017-04-25 14:12:49,705
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> User['metron']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,705
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['metron']
>> > >
>> > > 2017-04-25 14:12:49,760
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['livy']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,760
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['livy']
>> > >
>> > > 2017-04-25 14:12:49,803
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
True,
>> > > 'groups': [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,804
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['elasticsearch']
>> > >
>> > > 2017-04-25 14:12:49,837
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
User['spark']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,837
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['spark']
>> > >
>> > > 2017-04-25 14:12:49,885
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > User['ambari-qa']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
[u'users']}
>> > >
>> > > 2017-04-25 14:12:49,885
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['ambari-qa']
>> > >
>> > > 2017-04-25 14:12:49,928
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
User['flume']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,928
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['flume']
>> > >
>> > > 2017-04-25 14:12:49,972
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
User['kafka']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,973
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['kafka']
>> > >
>> > > 2017-04-25 14:12:50,025
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:50,026
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
>> > > User['hdfs']
>> > >
>> > > 2017-04-25 14:12:50,075
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['yarn']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:50,075
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
>> > > User['yarn']
>> > >
>> > > 2017-04-25 14:12:50,125
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> User['kibana']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:50,126
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
>> > > User['kibana']
>> > >
>> > > 2017-04-25 14:12:50,175
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> User['mapred']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:50,176
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
>> > > User['mapred']
>> > >
>> > > 2017-04-25 14:12:50,223
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
User['hbase']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:50,224
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
>> > > User['hbase']
>> > >
>> > > 2017-04-25 14:12:50,271
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hcat']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:50,272
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
>> > > User['hcat']
>> > >
>> > > 2017-04-25 14:12:50,320
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
>> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
>> > >
>> > > 2017-04-25 14:12:50,333
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
>> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
>> exist
>> > >
>> > > 2017-04-25 14:12:50,334
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
>> > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to
555
>> > >
>> > > 2017-04-25 14:12:50,335
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
>> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
>> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
>> $(id
>> > -u
>> > > ambari-qa) -gt 1000) || (false)'}
>> > >
>> > > 2017-04-25 14:12:50,364
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
>> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
>> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
>> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
>> > >
>> > > 2017-04-25 14:12:50,365
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
>> True,
>> > > 'mode': 0775, 'cd_access': 'a'}
>> > >
>> > > 2017-04-25 14:12:50,366
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
>> > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
>> > >
>> > > 2017-04-25 14:12:50,366
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
>> owner
>> > > for /tmp/hbase-hbase from 0 to hbase
>> > >
>> > > 2017-04-25 14:12:50,366
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
>> > > permission for /tmp/hbase-hbase from 755 to 775
>> > >
>> > > 2017-04-25 14:12:50,368
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
>> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
>> > >
>> > > 2017-04-25 14:12:50,369
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
>> > >
>/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>>
>> > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
>> > >
>> > > 2017-04-25 14:12:50,393
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
>> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
>> > >
>/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>>
>> > > due to not_if
>> > >
>> > > 2017-04-25 14:12:50,394
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Group['hdfs']
>> {}
>> > >
>> > > 2017-04-25 14:12:50,395
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding group
>> > > Group['hdfs']
>> > >
>> > > 2017-04-25 14:12:50,430
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
>> > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
>> > >
>> > > 2017-04-25 14:12:50,431
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
>> user
>> > > hdfs
>> > >
>> > > 2017-04-25 14:12:50,474
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
>> > >
>> > > 2017-04-25 14:12:50,474
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > Directory['/etc/hadoop'] {'mode': 0755}
>> > >
>> > > 2017-04-25 14:12:50,474
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
>> > > directory Directory['/etc/hadoop'] since it doesn't exist.
>> > >
>> > > 2017-04-25 14:12:50,475
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
>> {'owner':
>> > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
>> > >
>> > > 2017-04-25 14:12:50,476
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
>> > > directory
Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
>>
>> > > since it doesn't exist.
>> > >
>> > > 2017-04-25 14:12:50,476
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
>> owner
>> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
>> > >
>> > > 2017-04-25 14:12:50,476
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
>> group
>> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hadoop
>> > >
>> > > 2017-04-25 14:12:50,477
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
>> > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
>> 755
>> > > to 1777
>> > >
>> > > 2017-04-25 14:12:50,477
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
>> > > {'create_parents': True}
>> > >
>> > > 2017-04-25 14:12:50,478
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
>> > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
>> since
>> > > it doesn't exist.
>> > >
>> > > 2017-04-25 14:12:50,479
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
>> {'content':
>> > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
>> > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
>> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
>> > >
>> > > 2017-04-25 14:12:50,504
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Downloading
>> the
>> > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
>> > > linux-x64.tar.gz
>> > >
>> > > 2017-04-25 14:12:53,793
>> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>> > > Directory['/usr/jdk64'] {}
>> > >
>> > > 2017-04-25 14:12:53,793
>> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
>> > > directory Directory['/usr/jdk64'] since it doesn't exist.
>> > >
>> > > 2017-04-25 14:12:53,794
>> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>> > Execute[('chmod',
>> > > 'a+x', u'/usr/jdk64')] {'sudo': True}
>> > >
>> > > 2017-04-25 14:12:53,824
>> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute['cd
>> > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
>> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
ambari-sudo.sh
>> cp
>> > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
>> > >
>> > > 2017-04-25 14:13:00,439
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
>> > > ['delete']}
>> > >
>> > > 2017-04-25 14:13:00,440
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
>> > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and
>> all
>> > > its content
>> > >
>> > > 2017-04-25 14:13:00,620
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access':
>> 'a'}
>> > >
>> > > 2017-04-25 14:13:00,621
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > Execute[('chmod',
>> > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
>> > >
>> > > 2017-04-25 14:13:00,682
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Initializing
>> 2
>> > > repositories
>> > >
>> > > 2017-04-25 14:13:00,683
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
>> > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
>> ['create'],
>> > > 'components': [u'HDP', 'main'], 'repo_template':
>> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
>> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
>> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
>> > > 'mirror_list': None}
>> > >
>> > > 2017-04-25 14:13:00,709
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
>> > >
>> > > 2017-04-25 14:13:00,711
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
>> > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
>> > >
>> > > 2017-04-25 14:13:00,711
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://public-repo-1.
>> > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
>> ['create'],
>> > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
>> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
>> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
>> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS',
>> > > 'mirror_list': None}
>> > >
>> > > 2017-04-25 14:13:00,717
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
>> InlineTemplate(...)}
>> > >
>> > > 2017-04-25 14:13:00,717
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
>> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
>> > >
>> > > 2017-04-25 14:13:00,718
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> Package['unzip']
>> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
>> > >
>> > > 2017-04-25 14:13:00,846
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
>> > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
>> > >
>> > > 2017-04-25 14:13:03,413
>> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>> Package['curl']
>> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
>> > >
>> > > 2017-04-25 14:13:03,429
>> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
>> > > installation of existing package curl
>> > >
>> > > 2017-04-25 14:13:03,430
>> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>> > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
>> > > 'retry_count': 5}
>> > >
>> > > 2017-04-25 14:13:03,451
>> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
>> > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
>> > >
>> > > 2017-04-25 14:13:04,948
>> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES
>> Data
>> > > Node
>> > >
>> > > 2017-04-25 14:13:04,952
>> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
>> > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
>> False,
>> > > 'retry_count': 5}
>> > >
>> > > 2017-04-25 14:13:05,086
>> > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
>> > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
>> > > elasticsearch-2.3.3')
>> > >
>> > >
>> > >
>> > > Command failed after 1 tries
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On April 25, 2017 at 13:47:17, Otto Fowler (ottobackwards@gmail.com)
>> > > wrote:
>> > >
>> > > Yes, I’m going to run through again and get the logs. The basic
issue
>> > is,
>> > > no repo tab -> if I should have one,
>> > > and the ES yum command failing with nothing to do, which stops
>> deployment
>> > > on all the nodes.
>> > >
>> > >
>> > >
>> > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
>> > > michael.miklavcic@gmail.com) wrote:
>> > >
>> > > I've been doing an internal hwx deployment (HCP) recently for
testing
>> > > purposes. I have some ES tweaks to work out, but other than that it
>> has
>> > > been running smoothly. Can you share any log info or screenshots?
>> > >
>> > > Mike
>> > >
>> > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
>> wrote:
>> > >
>> > > > Hi,
>> > > > Should I see the repo tab in the ambari wizard for metron at this
>> > point?
>> > > > Asking for a friend who can’t get the wizard to work :)
>> > > >
>> > >
>> > >
>> >
>>
>>
Re: Ambari Wizard: Repo Tab
Posted by Anand Subramanian <as...@hortonworks.com>.
FWIW, I don’t see the Repository tab as well in recent builds.
Here's a screenshot of how it was earlier:
https://community.hortonworks.com/storage/attachments/9844-metron-screen2.png
And here's one from a recent build:
http://imgur.com/yjkltES
Regards,
Anand
On 4/26/17, 5:57 PM, "Otto Fowler" <ot...@gmail.com> wrote:
>Can you describe what ES issues you are working on?
>Fixing the repos got everything installed, but my ES components don’t stay
>running.
>I need to harvest the errors.
>
>
>
>On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com) wrote:
>
>Nm. sorry. I fixed it.
>
>
>
>On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com) wrote:
>
>Ok, now I see the repos in the ‘pick version’ screen, but it is erring on
>the f://localrepo
>even though the folder exists, there is no repodata/repomd.xml.
>
>What is the command to create a local repo?
>
>
>
>On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com) wrote:
>
>I was going by the HW community page.
>
>Ok, Let me try it
>
>
>
>On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com) wrote:
>
>That would do it. It requires 2.4.2+. I would have sworn I put that in the
>README, but I must have only annotated the PR. :(
>
>I'll get that in asap.
>
>-D...
>
>
>On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
>wrote:
>
>> 2.4.1 as a matter of fact
>>
>>
>> On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com) wrote:
>>
>> Any chance they're running Ambari < 2.4.2?
>>
>> -D...
>>
>>
>> On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
>> michael.miklavcic@gmail.com> wrote:
>>
>> > Hey Otto, I don't have wizard screenshots in front of me right now to
>> say
>> > for sure, but I do see a repoinfo.xml in the mpack. I haven't run into
>> > anything like that, but next time I run through the install I can take
>> > another look.
>> >
>> > https://github.com/apache/incubator-metron/blob/master/
>> > metron-deployment/packaging/ambari/metron-mpack/src/main/
>> > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
>> >
>> >
>> > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ot...@gmail.com>
>> > wrote:
>> >
>> > > stderr:
>> > >
>> > > Traceback (most recent call last):
>> > >
>> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
>> > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
>> > >
>> > > Elasticsearch().execute()
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > libraries/script/script.py",
>> > > line 280, in execute
>> > >
>> > > method(env)
>> > >
>> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
>> > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
>> > >
>> > > self.install_packages(env)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > libraries/script/script.py",
>> > > line 567, in install_packages
>> > >
>> > > retry_count=agent_stack_retry_count)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > core/base.py",
>> > > line 155, in __init__
>> > >
>> > > self.env.run()
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > core/environment.py",
>> > > line 160, in run
>> > >
>> > > self.run_action(resource, action)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > core/environment.py",
>> > > line 124, in run_action
>> > >
>> > > provider_action()
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > core/providers/package/__init__.py", line 54, in action_install
>> > >
>> > > self.install_package(package_name, self.resource.use_repos,
>> > > self.resource.skip_repos)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > core/providers/package/yumrpm.py", line 49, in install_package
>> > >
>> > > self.checked_call_with_retries(cmd, sudo=True,
>> > > logoutput=self.get_logoutput())
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > core/providers/package/__init__.py", line 83, in
>> > checked_call_with_retries
>> > >
>> > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > > core/providers/package/__init__.py", line 91, in _call_with_retries
>> > >
>> > > code, out = func(cmd, **kwargs)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > core/shell.py",
>> > > line 71, in inner
>> > >
>> > > result = function(command, **kwargs)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > core/shell.py",
>> > > line 93, in checked_call
>> > >
>> > > tries=tries, try_sleep=try_sleep)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > core/shell.py",
>> > > line 141, in _call_wrapper
>> > >
>> > > result = _call(command, **kwargs_copy)
>> > >
>> > > File "/usr/lib/python2.6/site-packages/resource_management/
>> > core/shell.py",
>> > > line 294, in _call
>> > >
>> > > raise Fail(err_msg)
>> > >
>> > > resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum
>> -d
>> > 0
>> > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to do
>> > >
>> > > stdout:
>> > >
>> > > 2017-04-25 14:12:48,669
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using hadoop
>> > conf
>> > > dir: /usr/hdp/current/hadoop-client/conf
>> > >
>> > > 2017-04-25 14:12:48,671
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Group['metron']
>> > {}
>> > >
>> > > 2017-04-25 14:12:48,673
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
>> > > Group['metron']
>> > >
>> > > 2017-04-25 14:12:48,740
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['livy']
>> {}
>> > >
>> > > 2017-04-25 14:12:48,741
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
>> > > Group['livy']
>> > >
>> > > 2017-04-25 14:12:48,774
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> > > Group['elasticsearch'] {}
>> > >
>> > > 2017-04-25 14:12:48,775
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
>> > > Group['elasticsearch']
>> > >
>> > > 2017-04-25 14:12:48,812
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Group['spark']
>> > {}
>> > >
>> > > 2017-04-25 14:12:48,812
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
>> > > Group['spark']
>> > >
>> > > 2017-04-25 14:12:48,848
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> > Group['zeppelin']
>> > > {}
>> > >
>> > > 2017-04-25 14:12:48,849
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
>> > > Group['zeppelin']
>> > >
>> > > 2017-04-25 14:12:48,884
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Group['hadoop']
>> > {}
>> > >
>> > > 2017-04-25 14:12:48,885
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
>> > > Group['hadoop']
>> > >
>> > > 2017-04-25 14:12:48,921
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Group['kibana']
>> > {}
>> > >
>> > > 2017-04-25 14:12:48,922
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
>> > > Group['kibana']
>> > >
>> > > 2017-04-25 14:12:48,959
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
>> Group['users']
>> > {}
>> > >
>> > > 2017-04-25 14:12:48,960
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - User['hive']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:48,961
>> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding user
>> > > User['hive']
>> > >
>> > > 2017-04-25 14:12:49,227
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['storm']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,227
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['storm']
>> > >
>> > > 2017-04-25 14:12:49,388
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > User['zookeeper']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,388
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['zookeeper']
>> > >
>> > > 2017-04-25 14:12:49,518
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
>> > > 'groups': [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,518
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['infra-solr']
>> > >
>> > > 2017-04-25 14:12:49,578
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ams']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,578
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['ams']
>> > >
>> > > 2017-04-25 14:12:49,625
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['tez']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
>> > >
>> > > 2017-04-25 14:12:49,626
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['tez']
>> > >
>> > > 2017-04-25 14:12:49,660
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> User['zeppelin']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,660
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['zeppelin']
>> > >
>> > > 2017-04-25 14:12:49,705
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> User['metron']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,705
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['metron']
>> > >
>> > > 2017-04-25 14:12:49,760
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['livy']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,760
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['livy']
>> > >
>> > > 2017-04-25 14:12:49,803
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
>> > > 'groups': [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,804
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['elasticsearch']
>> > >
>> > > 2017-04-25 14:12:49,837
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['spark']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,837
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['spark']
>> > >
>> > > 2017-04-25 14:12:49,885
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
>> > User['ambari-qa']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
>> > >
>> > > 2017-04-25 14:12:49,885
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['ambari-qa']
>> > >
>> > > 2017-04-25 14:12:49,928
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['flume']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,928
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['flume']
>> > >
>> > > 2017-04-25 14:12:49,972
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['kafka']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:49,973
>> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
>> > > User['kafka']
>> > >
>> > > 2017-04-25 14:12:50,025
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:50,026
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
>> > > User['hdfs']
>> > >
>> > > 2017-04-25 14:12:50,075
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['yarn']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:50,075
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
>> > > User['yarn']
>> > >
>> > > 2017-04-25 14:12:50,125
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> User['kibana']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:50,126
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
>> > > User['kibana']
>> > >
>> > > 2017-04-25 14:12:50,175
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> User['mapred']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:50,176
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
>> > > User['mapred']
>> > >
>> > > 2017-04-25 14:12:50,223
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hbase']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:50,224
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
>> > > User['hbase']
>> > >
>> > > 2017-04-25 14:12:50,271
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hcat']
>> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
>> [u'hadoop']}
>> > >
>> > > 2017-04-25 14:12:50,272
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
>> > > User['hcat']
>> > >
>> > > 2017-04-25 14:12:50,320
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
>> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
>> > >
>> > > 2017-04-25 14:12:50,333
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
>> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
>> exist
>> > >
>> > > 2017-04-25 14:12:50,334
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
>> > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to 555
>> > >
>> > > 2017-04-25 14:12:50,335
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
>> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
>> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
>> $(id
>> > -u
>> > > ambari-qa) -gt 1000) || (false)'}
>> > >
>> > > 2017-04-25 14:12:50,364
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
>> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
>> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
>> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
>> > >
>> > > 2017-04-25 14:12:50,365
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
>> True,
>> > > 'mode': 0775, 'cd_access': 'a'}
>> > >
>> > > 2017-04-25 14:12:50,366
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
>> > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
>> > >
>> > > 2017-04-25 14:12:50,366
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
>> owner
>> > > for /tmp/hbase-hbase from 0 to hbase
>> > >
>> > > 2017-04-25 14:12:50,366
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
>> > > permission for /tmp/hbase-hbase from 755 to 775
>> > >
>> > > 2017-04-25 14:12:50,368
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
>> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
>> > >
>> > > 2017-04-25 14:12:50,369
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
>> > >
>/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>>
>> > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
>> > >
>> > > 2017-04-25 14:12:50,393
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
>> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
>> > >
>/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>>
>> > > due to not_if
>> > >
>> > > 2017-04-25 14:12:50,394
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Group['hdfs']
>> {}
>> > >
>> > > 2017-04-25 14:12:50,395
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding group
>> > > Group['hdfs']
>> > >
>> > > 2017-04-25 14:12:50,430
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
>> > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
>> > >
>> > > 2017-04-25 14:12:50,431
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
>> user
>> > > hdfs
>> > >
>> > > 2017-04-25 14:12:50,474
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
>> > >
>> > > 2017-04-25 14:12:50,474
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > Directory['/etc/hadoop'] {'mode': 0755}
>> > >
>> > > 2017-04-25 14:12:50,474
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
>> > > directory Directory['/etc/hadoop'] since it doesn't exist.
>> > >
>> > > 2017-04-25 14:12:50,475
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
>> {'owner':
>> > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
>> > >
>> > > 2017-04-25 14:12:50,476
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
>> > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
>>
>> > > since it doesn't exist.
>> > >
>> > > 2017-04-25 14:12:50,476
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
>> owner
>> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
>> > >
>> > > 2017-04-25 14:12:50,476
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
>> group
>> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hadoop
>> > >
>> > > 2017-04-25 14:12:50,477
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
>> > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
>> 755
>> > > to 1777
>> > >
>> > > 2017-04-25 14:12:50,477
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
>> > > {'create_parents': True}
>> > >
>> > > 2017-04-25 14:12:50,478
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
>> > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
>> since
>> > > it doesn't exist.
>> > >
>> > > 2017-04-25 14:12:50,479
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
>> > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
>> {'content':
>> > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
>> > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
>> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
>> > >
>> > > 2017-04-25 14:12:50,504
>> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Downloading
>> the
>> > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
>> > > linux-x64.tar.gz
>> > >
>> > > 2017-04-25 14:12:53,793
>> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>> > > Directory['/usr/jdk64'] {}
>> > >
>> > > 2017-04-25 14:12:53,793
>> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
>> > > directory Directory['/usr/jdk64'] since it doesn't exist.
>> > >
>> > > 2017-04-25 14:12:53,794
>> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
>> > Execute[('chmod',
>> > > 'a+x', u'/usr/jdk64')] {'sudo': True}
>> > >
>> > > 2017-04-25 14:12:53,824
>> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute['cd
>> > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
>> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz && ambari-sudo.sh
>> cp
>> > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
>> > >
>> > > 2017-04-25 14:13:00,439
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
>> > > ['delete']}
>> > >
>> > > 2017-04-25 14:13:00,440
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
>> > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and
>> all
>> > > its content
>> > >
>> > > 2017-04-25 14:13:00,620
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access':
>> 'a'}
>> > >
>> > > 2017-04-25 14:13:00,621
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > Execute[('chmod',
>> > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
>> > >
>> > > 2017-04-25 14:13:00,682
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Initializing
>> 2
>> > > repositories
>> > >
>> > > 2017-04-25 14:13:00,683
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
>> > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
>> ['create'],
>> > > 'components': [u'HDP', 'main'], 'repo_template':
>> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
>> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
>> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
>> > > 'mirror_list': None}
>> > >
>> > > 2017-04-25 14:13:00,709
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
>> > >
>> > > 2017-04-25 14:13:00,711
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
>> > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
>> > >
>> > > 2017-04-25 14:13:00,711
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://public-repo-1.
>> > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
>> ['create'],
>> > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
>> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
>> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
>> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS',
>> > > 'mirror_list': None}
>> > >
>> > > 2017-04-25 14:13:00,717
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
>> InlineTemplate(...)}
>> > >
>> > > 2017-04-25 14:13:00,717
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
>> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
>> > >
>> > > 2017-04-25 14:13:00,718
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
>> Package['unzip']
>> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
>> > >
>> > > 2017-04-25 14:13:00,846
>> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
>> > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
>> > >
>> > > 2017-04-25 14:13:03,413
>> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>> Package['curl']
>> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
>> > >
>> > > 2017-04-25 14:13:03,429
>> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
>> > > installation of existing package curl
>> > >
>> > > 2017-04-25 14:13:03,430
>> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
>> > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
>> > > 'retry_count': 5}
>> > >
>> > > 2017-04-25 14:13:03,451
>> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
>> > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
>> > >
>> > > 2017-04-25 14:13:04,948
>> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES
>> Data
>> > > Node
>> > >
>> > > 2017-04-25 14:13:04,952
>> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
>> > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
>> False,
>> > > 'retry_count': 5}
>> > >
>> > > 2017-04-25 14:13:05,086
>> > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
>> > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
>> > > elasticsearch-2.3.3')
>> > >
>> > >
>> > >
>> > > Command failed after 1 tries
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On April 25, 2017 at 13:47:17, Otto Fowler (ottobackwards@gmail.com)
>> > > wrote:
>> > >
>> > > Yes, I’m going to run through again and get the logs. The basic issue
>> > is,
>> > > no repo tab -> if I should have one,
>> > > and the ES yum command failing with nothing to do, which stops
>> deployment
>> > > on all the nodes.
>> > >
>> > >
>> > >
>> > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
>> > > michael.miklavcic@gmail.com) wrote:
>> > >
>> > > I've been doing an internal hwx deployment (HCP) recently for testing
>> > > purposes. I have some ES tweaks to work out, but other than that it
>> has
>> > > been running smoothly. Can you share any log info or screenshots?
>> > >
>> > > Mike
>> > >
>> > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
>> wrote:
>> > >
>> > > > Hi,
>> > > > Should I see the repo tab in the ambari wizard for metron at this
>> > point?
>> > > > Asking for a friend who can’t get the wizard to work :)
>> > > >
>> > >
>> > >
>> >
>>
>>
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
Can you describe what ES issues you are working on?
Fixing the repos got everything installed, but my ES components don’t stay
running.
I need to harvest the errors.
On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwards@gmail.com) wrote:
Nm. sorry. I fixed it.
On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com) wrote:
Ok, now I see the repos in the ‘pick version’ screen, but it is erring on
the f://localrepo
even though the folder exists, there is no repodata/repomd.xml.
What is the command to create a local repo?
On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com) wrote:
I was going by the HW community page.
Ok, Let me try it
On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com) wrote:
That would do it. It requires 2.4.2+. I would have sworn I put that in the
README, but I must have only annotated the PR. :(
I'll get that in asap.
-D...
On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
wrote:
> 2.4.1 as a matter of fact
>
>
> On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com) wrote:
>
> Any chance they're running Ambari < 2.4.2?
>
> -D...
>
>
> On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> michael.miklavcic@gmail.com> wrote:
>
> > Hey Otto, I don't have wizard screenshots in front of me right now to
> say
> > for sure, but I do see a repoinfo.xml in the mpack. I haven't run into
> > anything like that, but next time I run through the install I can take
> > another look.
> >
> > https://github.com/apache/incubator-metron/blob/master/
> > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> >
> >
> > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > stderr:
> > >
> > > Traceback (most recent call last):
> > >
> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > >
> > > Elasticsearch().execute()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > libraries/script/script.py",
> > > line 280, in execute
> > >
> > > method(env)
> > >
> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > >
> > > self.install_packages(env)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > libraries/script/script.py",
> > > line 567, in install_packages
> > >
> > > retry_count=agent_stack_retry_count)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/base.py",
> > > line 155, in __init__
> > >
> > > self.env.run()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/environment.py",
> > > line 160, in run
> > >
> > > self.run_action(resource, action)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/environment.py",
> > > line 124, in run_action
> > >
> > > provider_action()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 54, in action_install
> > >
> > > self.install_package(package_name, self.resource.use_repos,
> > > self.resource.skip_repos)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/yumrpm.py", line 49, in install_package
> > >
> > > self.checked_call_with_retries(cmd, sudo=True,
> > > logoutput=self.get_logoutput())
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 83, in
> > checked_call_with_retries
> > >
> > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 91, in _call_with_retries
> > >
> > > code, out = func(cmd, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 71, in inner
> > >
> > > result = function(command, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 93, in checked_call
> > >
> > > tries=tries, try_sleep=try_sleep)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 141, in _call_wrapper
> > >
> > > result = _call(command, **kwargs_copy)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 294, in _call
> > >
> > > raise Fail(err_msg)
> > >
> > > resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum
> -d
> > 0
> > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to do
> > >
> > > stdout:
> > >
> > > 2017-04-25 14:12:48,669
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using hadoop
> > conf
> > > dir: /usr/hdp/current/hadoop-client/conf
> > >
> > > 2017-04-25 14:12:48,671
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['metron']
> > {}
> > >
> > > 2017-04-25 14:12:48,673
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['metron']
> > >
> > > 2017-04-25 14:12:48,740
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['livy']
> {}
> > >
> > > 2017-04-25 14:12:48,741
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['livy']
> > >
> > > 2017-04-25 14:12:48,774
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['elasticsearch'] {}
> > >
> > > 2017-04-25 14:12:48,775
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['elasticsearch']
> > >
> > > 2017-04-25 14:12:48,812
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['spark']
> > {}
> > >
> > > 2017-04-25 14:12:48,812
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['spark']
> > >
> > > 2017-04-25 14:12:48,848
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['zeppelin']
> > > {}
> > >
> > > 2017-04-25 14:12:48,849
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['zeppelin']
> > >
> > > 2017-04-25 14:12:48,884
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['hadoop']
> > {}
> > >
> > > 2017-04-25 14:12:48,885
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['hadoop']
> > >
> > > 2017-04-25 14:12:48,921
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['kibana']
> > {}
> > >
> > > 2017-04-25 14:12:48,922
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['kibana']
> > >
> > > 2017-04-25 14:12:48,959
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['users']
> > {}
> > >
> > > 2017-04-25 14:12:48,960
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - User['hive']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:48,961
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding user
> > > User['hive']
> > >
> > > 2017-04-25 14:12:49,227
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['storm']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,227
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['storm']
> > >
> > > 2017-04-25 14:12:49,388
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['zookeeper']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,388
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['zookeeper']
> > >
> > > 2017-04-25 14:12:49,518
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > > 'groups': [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,518
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['infra-solr']
> > >
> > > 2017-04-25 14:12:49,578
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ams']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,578
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['ams']
> > >
> > > 2017-04-25 14:12:49,625
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['tez']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> > >
> > > 2017-04-25 14:12:49,626
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['tez']
> > >
> > > 2017-04-25 14:12:49,660
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['zeppelin']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,660
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['zeppelin']
> > >
> > > 2017-04-25 14:12:49,705
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['metron']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,705
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['metron']
> > >
> > > 2017-04-25 14:12:49,760
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['livy']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,760
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['livy']
> > >
> > > 2017-04-25 14:12:49,803
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > > 'groups': [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,804
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['elasticsearch']
> > >
> > > 2017-04-25 14:12:49,837
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['spark']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,837
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['spark']
> > >
> > > 2017-04-25 14:12:49,885
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['ambari-qa']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> > >
> > > 2017-04-25 14:12:49,885
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['ambari-qa']
> > >
> > > 2017-04-25 14:12:49,928
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['flume']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,928
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['flume']
> > >
> > > 2017-04-25 14:12:49,972
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['kafka']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,973
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['kafka']
> > >
> > > 2017-04-25 14:12:50,025
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,026
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hdfs']
> > >
> > > 2017-04-25 14:12:50,075
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['yarn']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,075
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['yarn']
> > >
> > > 2017-04-25 14:12:50,125
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['kibana']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,126
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['kibana']
> > >
> > > 2017-04-25 14:12:50,175
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['mapred']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,176
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['mapred']
> > >
> > > 2017-04-25 14:12:50,223
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hbase']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,224
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hbase']
> > >
> > > 2017-04-25 14:12:50,271
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hcat']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,272
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hcat']
> > >
> > > 2017-04-25 14:12:50,320
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > >
> > > 2017-04-25 14:12:50,333
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
> exist
> > >
> > > 2017-04-25 14:12:50,334
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to 555
> > >
> > > 2017-04-25 14:12:50,335
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
> $(id
> > -u
> > > ambari-qa) -gt 1000) || (false)'}
> > >
> > > 2017-04-25 14:12:50,364
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > >
> > > 2017-04-25 14:12:50,365
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
> True,
> > > 'mode': 0775, 'cd_access': 'a'}
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> owner
> > > for /tmp/hbase-hbase from 0 to hbase
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /tmp/hbase-hbase from 755 to 775
> > >
> > > 2017-04-25 14:12:50,368
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > >
> > > 2017-04-25 14:12:50,369
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > >
/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>
> > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > >
> > > 2017-04-25 14:12:50,393
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > >
/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>
> > > due to not_if
> > >
> > > 2017-04-25 14:12:50,394
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Group['hdfs']
> {}
> > >
> > > 2017-04-25 14:12:50,395
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding group
> > > Group['hdfs']
> > >
> > > 2017-04-25 14:12:50,430
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > >
> > > 2017-04-25 14:12:50,431
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
> user
> > > hdfs
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/etc/hadoop'] {'mode': 0755}
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,475
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> {'owner':
> > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
>
> > > since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> owner
> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> group
> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hadoop
> > >
> > > 2017-04-25 14:12:50,477
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
> 755
> > > to 1777
> > >
> > > 2017-04-25 14:12:50,477
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > {'create_parents': True}
> > >
> > > 2017-04-25 14:12:50,478
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> since
> > > it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,479
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> {'content':
> > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > >
> > > 2017-04-25 14:12:50,504
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Downloading
> the
> > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > linux-x64.tar.gz
> > >
> > > 2017-04-25 14:12:53,793
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > Directory['/usr/jdk64'] {}
> > >
> > > 2017-04-25 14:12:53,793
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:53,794
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > Execute[('chmod',
> > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > >
> > > 2017-04-25 14:12:53,824
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute['cd
> > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz && ambari-sudo.sh
> cp
> > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > >
> > > 2017-04-25 14:13:00,439
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > ['delete']}
> > >
> > > 2017-04-25 14:13:00,440
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and
> all
> > > its content
> > >
> > > 2017-04-25 14:13:00,620
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access':
> 'a'}
> > >
> > > 2017-04-25 14:13:00,621
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Execute[('chmod',
> > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > >
> > > 2017-04-25 14:13:00,682
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Initializing
> 2
> > > repositories
> > >
> > > 2017-04-25 14:13:00,683
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> ['create'],
> > > 'components': [u'HDP', 'main'], 'repo_template':
> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > 'mirror_list': None}
> > >
> > > 2017-04-25 14:13:00,709
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
> > >
> > > 2017-04-25 14:13:00,711
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > >
> > > 2017-04-25 14:13:00,711
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://public-repo-1.
> > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> ['create'],
> > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS',
> > > 'mirror_list': None}
> > >
> > > 2017-04-25 14:13:00,717
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> InlineTemplate(...)}
> > >
> > > 2017-04-25 14:13:00,717
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > >
> > > 2017-04-25 14:13:00,718
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Package['unzip']
> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:00,846
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > >
> > > 2017-04-25 14:13:03,413
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> Package['curl']
> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:03,429
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > installation of existing package curl
> > >
> > > 2017-04-25 14:13:03,430
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:03,451
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
> > >
> > > 2017-04-25 14:13:04,948
> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES
> Data
> > > Node
> > >
> > > 2017-04-25 14:13:04,952
> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> False,
> > > 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:05,086
> > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > elasticsearch-2.3.3')
> > >
> > >
> > >
> > > Command failed after 1 tries
> > >
> > >
> > >
> > >
> > >
> > > On April 25, 2017 at 13:47:17, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > Yes, I’m going to run through again and get the logs. The basic issue
> > is,
> > > no repo tab -> if I should have one,
> > > and the ES yum command failing with nothing to do, which stops
> deployment
> > > on all the nodes.
> > >
> > >
> > >
> > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > michael.miklavcic@gmail.com) wrote:
> > >
> > > I've been doing an internal hwx deployment (HCP) recently for testing
> > > purposes. I have some ES tweaks to work out, but other than that it
> has
> > > been running smoothly. Can you share any log info or screenshots?
> > >
> > > Mike
> > >
> > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > > Should I see the repo tab in the ambari wizard for metron at this
> > point?
> > > > Asking for a friend who can’t get the wizard to work :)
> > > >
> > >
> > >
> >
>
>
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
Nm. sorry. I fixed it.
On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwards@gmail.com) wrote:
Ok, now I see the repos in the ‘pick version’ screen, but it is erring on
the f://localrepo
even though the folder exists, there is no repodata/repomd.xml.
What is the command to create a local repo?
On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com) wrote:
I was going by the HW community page.
Ok, Let me try it
On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com) wrote:
That would do it. It requires 2.4.2+. I would have sworn I put that in the
README, but I must have only annotated the PR. :(
I'll get that in asap.
-D...
On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
wrote:
> 2.4.1 as a matter of fact
>
>
> On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com) wrote:
>
> Any chance they're running Ambari < 2.4.2?
>
> -D...
>
>
> On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> michael.miklavcic@gmail.com> wrote:
>
> > Hey Otto, I don't have wizard screenshots in front of me right now to
> say
> > for sure, but I do see a repoinfo.xml in the mpack. I haven't run into
> > anything like that, but next time I run through the install I can take
> > another look.
> >
> > https://github.com/apache/incubator-metron/blob/master/
> > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> >
> >
> > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > stderr:
> > >
> > > Traceback (most recent call last):
> > >
> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > >
> > > Elasticsearch().execute()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > libraries/script/script.py",
> > > line 280, in execute
> > >
> > > method(env)
> > >
> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > >
> > > self.install_packages(env)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > libraries/script/script.py",
> > > line 567, in install_packages
> > >
> > > retry_count=agent_stack_retry_count)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/base.py",
> > > line 155, in __init__
> > >
> > > self.env.run()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/environment.py",
> > > line 160, in run
> > >
> > > self.run_action(resource, action)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/environment.py",
> > > line 124, in run_action
> > >
> > > provider_action()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 54, in action_install
> > >
> > > self.install_package(package_name, self.resource.use_repos,
> > > self.resource.skip_repos)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/yumrpm.py", line 49, in install_package
> > >
> > > self.checked_call_with_retries(cmd, sudo=True,
> > > logoutput=self.get_logoutput())
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 83, in
> > checked_call_with_retries
> > >
> > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 91, in _call_with_retries
> > >
> > > code, out = func(cmd, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 71, in inner
> > >
> > > result = function(command, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 93, in checked_call
> > >
> > > tries=tries, try_sleep=try_sleep)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 141, in _call_wrapper
> > >
> > > result = _call(command, **kwargs_copy)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 294, in _call
> > >
> > > raise Fail(err_msg)
> > >
> > > resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum
> -d
> > 0
> > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to do
> > >
> > > stdout:
> > >
> > > 2017-04-25 14:12:48,669
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using hadoop
> > conf
> > > dir: /usr/hdp/current/hadoop-client/conf
> > >
> > > 2017-04-25 14:12:48,671
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['metron']
> > {}
> > >
> > > 2017-04-25 14:12:48,673
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['metron']
> > >
> > > 2017-04-25 14:12:48,740
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['livy']
> {}
> > >
> > > 2017-04-25 14:12:48,741
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['livy']
> > >
> > > 2017-04-25 14:12:48,774
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['elasticsearch'] {}
> > >
> > > 2017-04-25 14:12:48,775
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['elasticsearch']
> > >
> > > 2017-04-25 14:12:48,812
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['spark']
> > {}
> > >
> > > 2017-04-25 14:12:48,812
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['spark']
> > >
> > > 2017-04-25 14:12:48,848
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['zeppelin']
> > > {}
> > >
> > > 2017-04-25 14:12:48,849
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['zeppelin']
> > >
> > > 2017-04-25 14:12:48,884
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['hadoop']
> > {}
> > >
> > > 2017-04-25 14:12:48,885
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['hadoop']
> > >
> > > 2017-04-25 14:12:48,921
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['kibana']
> > {}
> > >
> > > 2017-04-25 14:12:48,922
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['kibana']
> > >
> > > 2017-04-25 14:12:48,959
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['users']
> > {}
> > >
> > > 2017-04-25 14:12:48,960
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - User['hive']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:48,961
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding user
> > > User['hive']
> > >
> > > 2017-04-25 14:12:49,227
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['storm']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,227
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['storm']
> > >
> > > 2017-04-25 14:12:49,388
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['zookeeper']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,388
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['zookeeper']
> > >
> > > 2017-04-25 14:12:49,518
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > > 'groups': [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,518
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['infra-solr']
> > >
> > > 2017-04-25 14:12:49,578
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ams']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,578
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['ams']
> > >
> > > 2017-04-25 14:12:49,625
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['tez']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> > >
> > > 2017-04-25 14:12:49,626
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['tez']
> > >
> > > 2017-04-25 14:12:49,660
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['zeppelin']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,660
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['zeppelin']
> > >
> > > 2017-04-25 14:12:49,705
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['metron']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,705
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['metron']
> > >
> > > 2017-04-25 14:12:49,760
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['livy']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,760
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['livy']
> > >
> > > 2017-04-25 14:12:49,803
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > > 'groups': [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,804
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['elasticsearch']
> > >
> > > 2017-04-25 14:12:49,837
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['spark']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,837
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['spark']
> > >
> > > 2017-04-25 14:12:49,885
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['ambari-qa']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> > >
> > > 2017-04-25 14:12:49,885
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['ambari-qa']
> > >
> > > 2017-04-25 14:12:49,928
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['flume']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,928
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['flume']
> > >
> > > 2017-04-25 14:12:49,972
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['kafka']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,973
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['kafka']
> > >
> > > 2017-04-25 14:12:50,025
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,026
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hdfs']
> > >
> > > 2017-04-25 14:12:50,075
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['yarn']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,075
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['yarn']
> > >
> > > 2017-04-25 14:12:50,125
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['kibana']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,126
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['kibana']
> > >
> > > 2017-04-25 14:12:50,175
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['mapred']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,176
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['mapred']
> > >
> > > 2017-04-25 14:12:50,223
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hbase']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,224
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hbase']
> > >
> > > 2017-04-25 14:12:50,271
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hcat']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,272
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hcat']
> > >
> > > 2017-04-25 14:12:50,320
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > >
> > > 2017-04-25 14:12:50,333
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
> exist
> > >
> > > 2017-04-25 14:12:50,334
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to 555
> > >
> > > 2017-04-25 14:12:50,335
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
> $(id
> > -u
> > > ambari-qa) -gt 1000) || (false)'}
> > >
> > > 2017-04-25 14:12:50,364
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > >
> > > 2017-04-25 14:12:50,365
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
> True,
> > > 'mode': 0775, 'cd_access': 'a'}
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> owner
> > > for /tmp/hbase-hbase from 0 to hbase
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /tmp/hbase-hbase from 755 to 775
> > >
> > > 2017-04-25 14:12:50,368
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > >
> > > 2017-04-25 14:12:50,369
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > >
/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>
> > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > >
> > > 2017-04-25 14:12:50,393
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > >
/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>
> > > due to not_if
> > >
> > > 2017-04-25 14:12:50,394
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Group['hdfs']
> {}
> > >
> > > 2017-04-25 14:12:50,395
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding group
> > > Group['hdfs']
> > >
> > > 2017-04-25 14:12:50,430
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > >
> > > 2017-04-25 14:12:50,431
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
> user
> > > hdfs
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/etc/hadoop'] {'mode': 0755}
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,475
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> {'owner':
> > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
>
> > > since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> owner
> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> group
> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hadoop
> > >
> > > 2017-04-25 14:12:50,477
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
> 755
> > > to 1777
> > >
> > > 2017-04-25 14:12:50,477
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > {'create_parents': True}
> > >
> > > 2017-04-25 14:12:50,478
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> since
> > > it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,479
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> {'content':
> > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > >
> > > 2017-04-25 14:12:50,504
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Downloading
> the
> > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > linux-x64.tar.gz
> > >
> > > 2017-04-25 14:12:53,793
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > Directory['/usr/jdk64'] {}
> > >
> > > 2017-04-25 14:12:53,793
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:53,794
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > Execute[('chmod',
> > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > >
> > > 2017-04-25 14:12:53,824
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute['cd
> > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz && ambari-sudo.sh
> cp
> > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > >
> > > 2017-04-25 14:13:00,439
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > ['delete']}
> > >
> > > 2017-04-25 14:13:00,440
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and
> all
> > > its content
> > >
> > > 2017-04-25 14:13:00,620
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access':
> 'a'}
> > >
> > > 2017-04-25 14:13:00,621
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Execute[('chmod',
> > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > >
> > > 2017-04-25 14:13:00,682
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Initializing
> 2
> > > repositories
> > >
> > > 2017-04-25 14:13:00,683
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> ['create'],
> > > 'components': [u'HDP', 'main'], 'repo_template':
> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > 'mirror_list': None}
> > >
> > > 2017-04-25 14:13:00,709
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
> > >
> > > 2017-04-25 14:13:00,711
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > >
> > > 2017-04-25 14:13:00,711
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://public-repo-1.
> > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> ['create'],
> > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS',
> > > 'mirror_list': None}
> > >
> > > 2017-04-25 14:13:00,717
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> InlineTemplate(...)}
> > >
> > > 2017-04-25 14:13:00,717
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > >
> > > 2017-04-25 14:13:00,718
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Package['unzip']
> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:00,846
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > >
> > > 2017-04-25 14:13:03,413
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> Package['curl']
> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:03,429
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > installation of existing package curl
> > >
> > > 2017-04-25 14:13:03,430
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:03,451
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
> > >
> > > 2017-04-25 14:13:04,948
> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES
> Data
> > > Node
> > >
> > > 2017-04-25 14:13:04,952
> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> False,
> > > 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:05,086
> > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > elasticsearch-2.3.3')
> > >
> > >
> > >
> > > Command failed after 1 tries
> > >
> > >
> > >
> > >
> > >
> > > On April 25, 2017 at 13:47:17, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > Yes, I’m going to run through again and get the logs. The basic issue
> > is,
> > > no repo tab -> if I should have one,
> > > and the ES yum command failing with nothing to do, which stops
> deployment
> > > on all the nodes.
> > >
> > >
> > >
> > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > michael.miklavcic@gmail.com) wrote:
> > >
> > > I've been doing an internal hwx deployment (HCP) recently for testing
> > > purposes. I have some ES tweaks to work out, but other than that it
> has
> > > been running smoothly. Can you share any log info or screenshots?
> > >
> > > Mike
> > >
> > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > > Should I see the repo tab in the ambari wizard for metron at this
> > point?
> > > > Asking for a friend who can’t get the wizard to work :)
> > > >
> > >
> > >
> >
>
>
Re: Ambari Wizard: Repo Tab
Posted by Nick Allen <ni...@nickallen.org>.
createrepo
On Tue, Apr 25, 2017 at 4:42 PM, Otto Fowler <ot...@gmail.com>
wrote:
> Ok, now I see the repos in the ‘pick version’ screen, but it is erring on
> the f://localrepo
> even though the folder exists, there is no repodata/repomd.xml.
>
> What is the command to create a local repo?
>
>
>
> On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> I was going by the HW community page.
>
> Ok, Let me try it
>
>
>
> On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com) wrote:
>
> That would do it. It requires 2.4.2+. I would have sworn I put that in the
> README, but I must have only annotated the PR. :(
>
> I'll get that in asap.
>
> -D...
>
>
> On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > 2.4.1 as a matter of fact
> >
> >
> > On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com) wrote:
> >
> > Any chance they're running Ambari < 2.4.2?
> >
> > -D...
> >
> >
> > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> > michael.miklavcic@gmail.com> wrote:
> >
> > > Hey Otto, I don't have wizard screenshots in front of me right now to
> > say
> > > for sure, but I do see a repoinfo.xml in the mpack. I haven't run into
> > > anything like that, but next time I run through the install I can take
> > > another look.
> > >
> > > https://github.com/apache/incubator-metron/blob/master/
> > > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> > >
> > >
> > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ottobackwards@gmail.com
> >
> > > wrote:
> > >
> > > > stderr:
> > > >
> > > > Traceback (most recent call last):
> > > >
> > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > > >
> > > > Elasticsearch().execute()
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > libraries/script/script.py",
> > > > line 280, in execute
> > > >
> > > > method(env)
> > > >
> > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > > >
> > > > self.install_packages(env)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > libraries/script/script.py",
> > > > line 567, in install_packages
> > > >
> > > > retry_count=agent_stack_retry_count)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/base.py",
> > > > line 155, in __init__
> > > >
> > > > self.env.run()
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/environment.py",
> > > > line 160, in run
> > > >
> > > > self.run_action(resource, action)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/environment.py",
> > > > line 124, in run_action
> > > >
> > > > provider_action()
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/providers/package/__init__.py", line 54, in action_install
> > > >
> > > > self.install_package(package_name, self.resource.use_repos,
> > > > self.resource.skip_repos)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/providers/package/yumrpm.py", line 49, in install_package
> > > >
> > > > self.checked_call_with_retries(cmd, sudo=True,
> > > > logoutput=self.get_logoutput())
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/providers/package/__init__.py", line 83, in
> > > checked_call_with_retries
> > > >
> > > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > > core/providers/package/__init__.py", line 91, in _call_with_retries
> > > >
> > > > code, out = func(cmd, **kwargs)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/shell.py",
> > > > line 71, in inner
> > > >
> > > > result = function(command, **kwargs)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/shell.py",
> > > > line 93, in checked_call
> > > >
> > > > tries=tries, try_sleep=try_sleep)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/shell.py",
> > > > line 141, in _call_wrapper
> > > >
> > > > result = _call(command, **kwargs_copy)
> > > >
> > > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/shell.py",
> > > > line 294, in _call
> > > >
> > > > raise Fail(err_msg)
> > > >
> > > > resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum
> > -d
> > > 0
> > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to do
> > > >
> > > > stdout:
> > > >
> > > > 2017-04-25 14:12:48,669
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using hadoop
> > > conf
> > > > dir: /usr/hdp/current/hadoop-client/conf
> > > >
> > > > 2017-04-25 14:12:48,671
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['metron']
> > > {}
> > > >
> > > > 2017-04-25 14:12:48,673
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > > Group['metron']
> > > >
> > > > 2017-04-25 14:12:48,740
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['livy']
> > {}
> > > >
> > > > 2017-04-25 14:12:48,741
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > > Group['livy']
> > > >
> > > > 2017-04-25 14:12:48,774
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > > Group['elasticsearch'] {}
> > > >
> > > > 2017-04-25 14:12:48,775
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > > Group['elasticsearch']
> > > >
> > > > 2017-04-25 14:12:48,812
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['spark']
> > > {}
> > > >
> > > > 2017-04-25 14:12:48,812
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > > Group['spark']
> > > >
> > > > 2017-04-25 14:12:48,848
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['zeppelin']
> > > > {}
> > > >
> > > > 2017-04-25 14:12:48,849
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > > Group['zeppelin']
> > > >
> > > > 2017-04-25 14:12:48,884
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['hadoop']
> > > {}
> > > >
> > > > 2017-04-25 14:12:48,885
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > > Group['hadoop']
> > > >
> > > > 2017-04-25 14:12:48,921
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['kibana']
> > > {}
> > > >
> > > > 2017-04-25 14:12:48,922
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > > Group['kibana']
> > > >
> > > > 2017-04-25 14:12:48,959
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['users']
> > > {}
> > > >
> > > > 2017-04-25 14:12:48,960
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - User['hive']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:48,961
> > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding user
> > > > User['hive']
> > > >
> > > > 2017-04-25 14:12:49,227
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['storm']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,227
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['storm']
> > > >
> > > > 2017-04-25 14:12:49,388
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['zookeeper']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,388
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['zookeeper']
> > > >
> > > > 2017-04-25 14:12:49,518
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > > > 'groups': [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,518
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['infra-solr']
> > > >
> > > > 2017-04-25 14:12:49,578
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ams']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,578
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['ams']
> > > >
> > > > 2017-04-25 14:12:49,625
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['tez']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'users']}
> > > >
> > > > 2017-04-25 14:12:49,626
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['tez']
> > > >
> > > > 2017-04-25 14:12:49,660
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['zeppelin']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,660
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['zeppelin']
> > > >
> > > > 2017-04-25 14:12:49,705
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['metron']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,705
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['metron']
> > > >
> > > > 2017-04-25 14:12:49,760
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['livy']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,760
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['livy']
> > > >
> > > > 2017-04-25 14:12:49,803
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
> True,
> > > > 'groups': [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,804
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['elasticsearch']
> > > >
> > > > 2017-04-25 14:12:49,837
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['spark']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,837
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['spark']
> > > >
> > > > 2017-04-25 14:12:49,885
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['ambari-qa']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'users']}
> > > >
> > > > 2017-04-25 14:12:49,885
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['ambari-qa']
> > > >
> > > > 2017-04-25 14:12:49,928
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['flume']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,928
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['flume']
> > > >
> > > > 2017-04-25 14:12:49,972
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['kafka']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:49,973
> > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > > User['kafka']
> > > >
> > > > 2017-04-25 14:12:50,025
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,026
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['hdfs']
> > > >
> > > > 2017-04-25 14:12:50,075
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['yarn']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,075
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['yarn']
> > > >
> > > > 2017-04-25 14:12:50,125
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['kibana']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,126
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['kibana']
> > > >
> > > > 2017-04-25 14:12:50,175
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > User['mapred']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,176
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['mapred']
> > > >
> > > > 2017-04-25 14:12:50,223
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['hbase']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,224
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['hbase']
> > > >
> > > > 2017-04-25 14:12:50,271
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hcat']
> > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> > [u'hadoop']}
> > > >
> > > > 2017-04-25 14:12:50,272
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > > User['hcat']
> > > >
> > > > 2017-04-25 14:12:50,320
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > >
> > > > 2017-04-25 14:12:50,333
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
> > exist
> > > >
> > > > 2017-04-25 14:12:50,334
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to
> 555
> > > >
> > > > 2017-04-25 14:12:50,335
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
> > $(id
> > > -u
> > > > ambari-qa) -gt 1000) || (false)'}
> > > >
> > > > 2017-04-25 14:12:50,364
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > > >
> > > > 2017-04-25 14:12:50,365
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
> > True,
> > > > 'mode': 0775, 'cd_access': 'a'}
> > > >
> > > > 2017-04-25 14:12:50,366
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > > >
> > > > 2017-04-25 14:12:50,366
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > owner
> > > > for /tmp/hbase-hbase from 0 to hbase
> > > >
> > > > 2017-04-25 14:12:50,366
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > permission for /tmp/hbase-hbase from 755 to 775
> > > >
> > > > 2017-04-25 14:12:50,368
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > > >
> > > > 2017-04-25 14:12:50,369
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > >
> /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> >
> > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > > >
> > > > 2017-04-25 14:12:50,393
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > >
> /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> >
> > > > due to not_if
> > > >
> > > > 2017-04-25 14:12:50,394
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Group['hdfs']
> > {}
> > > >
> > > > 2017-04-25 14:12:50,395
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding group
> > > > Group['hdfs']
> > > >
> > > > 2017-04-25 14:12:50,430
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > > >
> > > > 2017-04-25 14:12:50,431
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
> > user
> > > > hdfs
> > > >
> > > > 2017-04-25 14:12:50,474
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > > >
> > > > 2017-04-25 14:12:50,474
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Directory['/etc/hadoop'] {'mode': 0755}
> > > >
> > > > 2017-04-25 14:12:50,474
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > > >
> > > > 2017-04-25 14:12:50,475
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > {'owner':
> > > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > > >
> > > > 2017-04-25 14:12:50,476
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_
> tmpdir']
> >
> > > > since it doesn't exist.
> > > >
> > > > 2017-04-25 14:12:50,476
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > owner
> > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
> > > >
> > > > 2017-04-25 14:12:50,476
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > group
> > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hadoop
> > > >
> > > > 2017-04-25 14:12:50,477
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
> > 755
> > > > to 1777
> > > >
> > > > 2017-04-25 14:12:50,477
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > > {'create_parents': True}
> > > >
> > > > 2017-04-25 14:12:50,478
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > since
> > > > it doesn't exist.
> > > >
> > > > 2017-04-25 14:12:50,479
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> > {'content':
> > > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > > >
> > > > 2017-04-25 14:12:50,504
> > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Downloading
> > the
> > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > > linux-x64.tar.gz
> > > >
> > > > 2017-04-25 14:12:53,793
> > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > > Directory['/usr/jdk64'] {}
> > > >
> > > > 2017-04-25 14:12:53,793
> > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > > >
> > > > 2017-04-25 14:12:53,794
> > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > Execute[('chmod',
> > > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > > >
> > > > 2017-04-25 14:12:53,824
> > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute['cd
> > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz &&
> ambari-sudo.sh
> > cp
> > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > > >
> > > > 2017-04-25 14:13:00,439
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > > ['delete']}
> > > >
> > > > 2017-04-25 14:13:00,440
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and
> > all
> > > > its content
> > > >
> > > > 2017-04-25 14:13:00,620
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access':
> > 'a'}
> > > >
> > > > 2017-04-25 14:13:00,621
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Execute[('chmod',
> > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > > >
> > > > 2017-04-25 14:13:00,682
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Initializing
> > 2
> > > > repositories
> > > >
> > > > 2017-04-25 14:13:00,683
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> > ['create'],
> > > > 'components': [u'HDP', 'main'], 'repo_template':
> > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > > 'mirror_list': None}
> > > >
> > > > 2017-04-25 14:13:00,709
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
> > > >
> > > > 2017-04-25 14:13:00,711
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > > >
> > > > 2017-04-25 14:13:00,711
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://public-repo-1.
> > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> > ['create'],
> > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS',
> > > > 'mirror_list': None}
> > > >
> > > > 2017-04-25 14:13:00,717
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> > InlineTemplate(...)}
> > > >
> > > > 2017-04-25 14:13:00,717
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > > >
> > > > 2017-04-25 14:13:00,718
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Package['unzip']
> > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > >
> > > > 2017-04-25 14:13:00,846
> > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > > >
> > > > 2017-04-25 14:13:03,413
> > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > Package['curl']
> > > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > > >
> > > > 2017-04-25 14:13:03,429
> > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > > installation of existing package curl
> > > >
> > > > 2017-04-25 14:13:03,430
> > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > > 'retry_count': 5}
> > > >
> > > > 2017-04-25 14:13:03,451
> > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
> > > >
> > > > 2017-04-25 14:13:04,948
> > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES
> > Data
> > > > Node
> > > >
> > > > 2017-04-25 14:13:04,952
> > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> > False,
> > > > 'retry_count': 5}
> > > >
> > > > 2017-04-25 14:13:05,086
> > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > > elasticsearch-2.3.3')
> > > >
> > > >
> > > >
> > > > Command failed after 1 tries
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 13:47:17, Otto Fowler (ottobackwards@gmail.com)
> > > > wrote:
> > > >
> > > > Yes, I’m going to run through again and get the logs. The basic issue
> > > is,
> > > > no repo tab -> if I should have one,
> > > > and the ES yum command failing with nothing to do, which stops
> > deployment
> > > > on all the nodes.
> > > >
> > > >
> > > >
> > > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > > michael.miklavcic@gmail.com) wrote:
> > > >
> > > > I've been doing an internal hwx deployment (HCP) recently for testing
> > > > purposes. I have some ES tweaks to work out, but other than that it
> > has
> > > > been running smoothly. Can you share any log info or screenshots?
> > > >
> > > > Mike
> > > >
> > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
> > wrote:
> > > >
> > > > > Hi,
> > > > > Should I see the repo tab in the ambari wizard for metron at this
> > > point?
> > > > > Asking for a friend who can’t get the wizard to work :)
> > > > >
> > > >
> > > >
> > >
> >
> >
>
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
Ok, now I see the repos in the ‘pick version’ screen, but it is erring on
the f://localrepo
even though the folder exists, there is no repodata/repomd.xml.
What is the command to create a local repo?
On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwards@gmail.com) wrote:
I was going by the HW community page.
Ok, Let me try it
On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com) wrote:
That would do it. It requires 2.4.2+. I would have sworn I put that in the
README, but I must have only annotated the PR. :(
I'll get that in asap.
-D...
On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
wrote:
> 2.4.1 as a matter of fact
>
>
> On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com) wrote:
>
> Any chance they're running Ambari < 2.4.2?
>
> -D...
>
>
> On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> michael.miklavcic@gmail.com> wrote:
>
> > Hey Otto, I don't have wizard screenshots in front of me right now to
> say
> > for sure, but I do see a repoinfo.xml in the mpack. I haven't run into
> > anything like that, but next time I run through the install I can take
> > another look.
> >
> > https://github.com/apache/incubator-metron/blob/master/
> > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> >
> >
> > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > stderr:
> > >
> > > Traceback (most recent call last):
> > >
> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > >
> > > Elasticsearch().execute()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > libraries/script/script.py",
> > > line 280, in execute
> > >
> > > method(env)
> > >
> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > >
> > > self.install_packages(env)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > libraries/script/script.py",
> > > line 567, in install_packages
> > >
> > > retry_count=agent_stack_retry_count)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/base.py",
> > > line 155, in __init__
> > >
> > > self.env.run()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/environment.py",
> > > line 160, in run
> > >
> > > self.run_action(resource, action)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/environment.py",
> > > line 124, in run_action
> > >
> > > provider_action()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 54, in action_install
> > >
> > > self.install_package(package_name, self.resource.use_repos,
> > > self.resource.skip_repos)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/yumrpm.py", line 49, in install_package
> > >
> > > self.checked_call_with_retries(cmd, sudo=True,
> > > logoutput=self.get_logoutput())
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 83, in
> > checked_call_with_retries
> > >
> > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 91, in _call_with_retries
> > >
> > > code, out = func(cmd, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 71, in inner
> > >
> > > result = function(command, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 93, in checked_call
> > >
> > > tries=tries, try_sleep=try_sleep)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 141, in _call_wrapper
> > >
> > > result = _call(command, **kwargs_copy)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 294, in _call
> > >
> > > raise Fail(err_msg)
> > >
> > > resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum
> -d
> > 0
> > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to do
> > >
> > > stdout:
> > >
> > > 2017-04-25 14:12:48,669
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using hadoop
> > conf
> > > dir: /usr/hdp/current/hadoop-client/conf
> > >
> > > 2017-04-25 14:12:48,671
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['metron']
> > {}
> > >
> > > 2017-04-25 14:12:48,673
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['metron']
> > >
> > > 2017-04-25 14:12:48,740
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['livy']
> {}
> > >
> > > 2017-04-25 14:12:48,741
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['livy']
> > >
> > > 2017-04-25 14:12:48,774
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['elasticsearch'] {}
> > >
> > > 2017-04-25 14:12:48,775
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['elasticsearch']
> > >
> > > 2017-04-25 14:12:48,812
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['spark']
> > {}
> > >
> > > 2017-04-25 14:12:48,812
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['spark']
> > >
> > > 2017-04-25 14:12:48,848
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['zeppelin']
> > > {}
> > >
> > > 2017-04-25 14:12:48,849
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['zeppelin']
> > >
> > > 2017-04-25 14:12:48,884
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['hadoop']
> > {}
> > >
> > > 2017-04-25 14:12:48,885
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['hadoop']
> > >
> > > 2017-04-25 14:12:48,921
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['kibana']
> > {}
> > >
> > > 2017-04-25 14:12:48,922
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['kibana']
> > >
> > > 2017-04-25 14:12:48,959
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['users']
> > {}
> > >
> > > 2017-04-25 14:12:48,960
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - User['hive']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:48,961
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding user
> > > User['hive']
> > >
> > > 2017-04-25 14:12:49,227
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['storm']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,227
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['storm']
> > >
> > > 2017-04-25 14:12:49,388
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['zookeeper']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,388
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['zookeeper']
> > >
> > > 2017-04-25 14:12:49,518
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > > 'groups': [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,518
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['infra-solr']
> > >
> > > 2017-04-25 14:12:49,578
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ams']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,578
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['ams']
> > >
> > > 2017-04-25 14:12:49,625
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['tez']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> > >
> > > 2017-04-25 14:12:49,626
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['tez']
> > >
> > > 2017-04-25 14:12:49,660
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['zeppelin']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,660
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['zeppelin']
> > >
> > > 2017-04-25 14:12:49,705
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['metron']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,705
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['metron']
> > >
> > > 2017-04-25 14:12:49,760
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['livy']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,760
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['livy']
> > >
> > > 2017-04-25 14:12:49,803
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > > 'groups': [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,804
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['elasticsearch']
> > >
> > > 2017-04-25 14:12:49,837
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['spark']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,837
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['spark']
> > >
> > > 2017-04-25 14:12:49,885
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['ambari-qa']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> > >
> > > 2017-04-25 14:12:49,885
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['ambari-qa']
> > >
> > > 2017-04-25 14:12:49,928
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['flume']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,928
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['flume']
> > >
> > > 2017-04-25 14:12:49,972
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['kafka']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,973
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['kafka']
> > >
> > > 2017-04-25 14:12:50,025
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,026
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hdfs']
> > >
> > > 2017-04-25 14:12:50,075
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['yarn']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,075
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['yarn']
> > >
> > > 2017-04-25 14:12:50,125
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['kibana']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,126
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['kibana']
> > >
> > > 2017-04-25 14:12:50,175
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['mapred']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,176
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['mapred']
> > >
> > > 2017-04-25 14:12:50,223
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hbase']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,224
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hbase']
> > >
> > > 2017-04-25 14:12:50,271
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hcat']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,272
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hcat']
> > >
> > > 2017-04-25 14:12:50,320
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > >
> > > 2017-04-25 14:12:50,333
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
> exist
> > >
> > > 2017-04-25 14:12:50,334
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to 555
> > >
> > > 2017-04-25 14:12:50,335
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
> $(id
> > -u
> > > ambari-qa) -gt 1000) || (false)'}
> > >
> > > 2017-04-25 14:12:50,364
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > >
> > > 2017-04-25 14:12:50,365
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
> True,
> > > 'mode': 0775, 'cd_access': 'a'}
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> owner
> > > for /tmp/hbase-hbase from 0 to hbase
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /tmp/hbase-hbase from 755 to 775
> > >
> > > 2017-04-25 14:12:50,368
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > >
> > > 2017-04-25 14:12:50,369
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > >
/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>
> > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > >
> > > 2017-04-25 14:12:50,393
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > >
/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>
> > > due to not_if
> > >
> > > 2017-04-25 14:12:50,394
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Group['hdfs']
> {}
> > >
> > > 2017-04-25 14:12:50,395
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding group
> > > Group['hdfs']
> > >
> > > 2017-04-25 14:12:50,430
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > >
> > > 2017-04-25 14:12:50,431
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
> user
> > > hdfs
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/etc/hadoop'] {'mode': 0755}
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,475
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> {'owner':
> > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
>
> > > since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> owner
> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> group
> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hadoop
> > >
> > > 2017-04-25 14:12:50,477
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
> 755
> > > to 1777
> > >
> > > 2017-04-25 14:12:50,477
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > {'create_parents': True}
> > >
> > > 2017-04-25 14:12:50,478
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> since
> > > it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,479
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> {'content':
> > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > >
> > > 2017-04-25 14:12:50,504
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Downloading
> the
> > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > linux-x64.tar.gz
> > >
> > > 2017-04-25 14:12:53,793
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > Directory['/usr/jdk64'] {}
> > >
> > > 2017-04-25 14:12:53,793
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:53,794
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > Execute[('chmod',
> > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > >
> > > 2017-04-25 14:12:53,824
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute['cd
> > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz && ambari-sudo.sh
> cp
> > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > >
> > > 2017-04-25 14:13:00,439
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > ['delete']}
> > >
> > > 2017-04-25 14:13:00,440
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and
> all
> > > its content
> > >
> > > 2017-04-25 14:13:00,620
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access':
> 'a'}
> > >
> > > 2017-04-25 14:13:00,621
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Execute[('chmod',
> > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > >
> > > 2017-04-25 14:13:00,682
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Initializing
> 2
> > > repositories
> > >
> > > 2017-04-25 14:13:00,683
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> ['create'],
> > > 'components': [u'HDP', 'main'], 'repo_template':
> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > 'mirror_list': None}
> > >
> > > 2017-04-25 14:13:00,709
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
> > >
> > > 2017-04-25 14:13:00,711
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > >
> > > 2017-04-25 14:13:00,711
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://public-repo-1.
> > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> ['create'],
> > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS',
> > > 'mirror_list': None}
> > >
> > > 2017-04-25 14:13:00,717
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> InlineTemplate(...)}
> > >
> > > 2017-04-25 14:13:00,717
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > >
> > > 2017-04-25 14:13:00,718
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Package['unzip']
> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:00,846
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > >
> > > 2017-04-25 14:13:03,413
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> Package['curl']
> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:03,429
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > installation of existing package curl
> > >
> > > 2017-04-25 14:13:03,430
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:03,451
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
> > >
> > > 2017-04-25 14:13:04,948
> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES
> Data
> > > Node
> > >
> > > 2017-04-25 14:13:04,952
> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> False,
> > > 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:05,086
> > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > elasticsearch-2.3.3')
> > >
> > >
> > >
> > > Command failed after 1 tries
> > >
> > >
> > >
> > >
> > >
> > > On April 25, 2017 at 13:47:17, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > Yes, I’m going to run through again and get the logs. The basic issue
> > is,
> > > no repo tab -> if I should have one,
> > > and the ES yum command failing with nothing to do, which stops
> deployment
> > > on all the nodes.
> > >
> > >
> > >
> > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > michael.miklavcic@gmail.com) wrote:
> > >
> > > I've been doing an internal hwx deployment (HCP) recently for testing
> > > purposes. I have some ES tweaks to work out, but other than that it
> has
> > > been running smoothly. Can you share any log info or screenshots?
> > >
> > > Mike
> > >
> > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > > Should I see the repo tab in the ambari wizard for metron at this
> > point?
> > > > Asking for a friend who can’t get the wizard to work :)
> > > >
> > >
> > >
> >
>
>
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
I was going by the HW community page.
Ok, Let me try it
On April 25, 2017 at 16:04:07, David Lyle (dlyle65535@gmail.com) wrote:
That would do it. It requires 2.4.2+. I would have sworn I put that in the
README, but I must have only annotated the PR. :(
I'll get that in asap.
-D...
On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
wrote:
> 2.4.1 as a matter of fact
>
>
> On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com) wrote:
>
> Any chance they're running Ambari < 2.4.2?
>
> -D...
>
>
> On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> michael.miklavcic@gmail.com> wrote:
>
> > Hey Otto, I don't have wizard screenshots in front of me right now to
> say
> > for sure, but I do see a repoinfo.xml in the mpack. I haven't run into
> > anything like that, but next time I run through the install I can take
> > another look.
> >
> > https://github.com/apache/incubator-metron/blob/master/
> > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> >
> >
> > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > stderr:
> > >
> > > Traceback (most recent call last):
> > >
> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > >
> > > Elasticsearch().execute()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > libraries/script/script.py",
> > > line 280, in execute
> > >
> > > method(env)
> > >
> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > >
> > > self.install_packages(env)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > libraries/script/script.py",
> > > line 567, in install_packages
> > >
> > > retry_count=agent_stack_retry_count)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/base.py",
> > > line 155, in __init__
> > >
> > > self.env.run()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/environment.py",
> > > line 160, in run
> > >
> > > self.run_action(resource, action)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/environment.py",
> > > line 124, in run_action
> > >
> > > provider_action()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 54, in action_install
> > >
> > > self.install_package(package_name, self.resource.use_repos,
> > > self.resource.skip_repos)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/yumrpm.py", line 49, in install_package
> > >
> > > self.checked_call_with_retries(cmd, sudo=True,
> > > logoutput=self.get_logoutput())
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 83, in
> > checked_call_with_retries
> > >
> > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 91, in _call_with_retries
> > >
> > > code, out = func(cmd, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 71, in inner
> > >
> > > result = function(command, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 93, in checked_call
> > >
> > > tries=tries, try_sleep=try_sleep)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 141, in _call_wrapper
> > >
> > > result = _call(command, **kwargs_copy)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 294, in _call
> > >
> > > raise Fail(err_msg)
> > >
> > > resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum
> -d
> > 0
> > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to do
> > >
> > > stdout:
> > >
> > > 2017-04-25 14:12:48,669
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using hadoop
> > conf
> > > dir: /usr/hdp/current/hadoop-client/conf
> > >
> > > 2017-04-25 14:12:48,671
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['metron']
> > {}
> > >
> > > 2017-04-25 14:12:48,673
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['metron']
> > >
> > > 2017-04-25 14:12:48,740
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['livy']
> {}
> > >
> > > 2017-04-25 14:12:48,741
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['livy']
> > >
> > > 2017-04-25 14:12:48,774
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['elasticsearch'] {}
> > >
> > > 2017-04-25 14:12:48,775
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['elasticsearch']
> > >
> > > 2017-04-25 14:12:48,812
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['spark']
> > {}
> > >
> > > 2017-04-25 14:12:48,812
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['spark']
> > >
> > > 2017-04-25 14:12:48,848
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['zeppelin']
> > > {}
> > >
> > > 2017-04-25 14:12:48,849
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['zeppelin']
> > >
> > > 2017-04-25 14:12:48,884
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['hadoop']
> > {}
> > >
> > > 2017-04-25 14:12:48,885
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['hadoop']
> > >
> > > 2017-04-25 14:12:48,921
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['kibana']
> > {}
> > >
> > > 2017-04-25 14:12:48,922
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['kibana']
> > >
> > > 2017-04-25 14:12:48,959
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['users']
> > {}
> > >
> > > 2017-04-25 14:12:48,960
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - User['hive']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:48,961
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding user
> > > User['hive']
> > >
> > > 2017-04-25 14:12:49,227
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['storm']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,227
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['storm']
> > >
> > > 2017-04-25 14:12:49,388
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['zookeeper']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,388
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['zookeeper']
> > >
> > > 2017-04-25 14:12:49,518
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > > 'groups': [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,518
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['infra-solr']
> > >
> > > 2017-04-25 14:12:49,578
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ams']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,578
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['ams']
> > >
> > > 2017-04-25 14:12:49,625
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['tez']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
[u'users']}
> > >
> > > 2017-04-25 14:12:49,626
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['tez']
> > >
> > > 2017-04-25 14:12:49,660
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['zeppelin']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,660
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['zeppelin']
> > >
> > > 2017-04-25 14:12:49,705
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['metron']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,705
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['metron']
> > >
> > > 2017-04-25 14:12:49,760
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['livy']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,760
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['livy']
> > >
> > > 2017-04-25 14:12:49,803
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups':
True,
> > > 'groups': [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,804
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['elasticsearch']
> > >
> > > 2017-04-25 14:12:49,837
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['spark']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,837
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['spark']
> > >
> > > 2017-04-25 14:12:49,885
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['ambari-qa']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
[u'users']}
> > >
> > > 2017-04-25 14:12:49,885
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['ambari-qa']
> > >
> > > 2017-04-25 14:12:49,928
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['flume']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,928
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['flume']
> > >
> > > 2017-04-25 14:12:49,972
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['kafka']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,973
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['kafka']
> > >
> > > 2017-04-25 14:12:50,025
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,026
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hdfs']
> > >
> > > 2017-04-25 14:12:50,075
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['yarn']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,075
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['yarn']
> > >
> > > 2017-04-25 14:12:50,125
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['kibana']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,126
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['kibana']
> > >
> > > 2017-04-25 14:12:50,175
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['mapred']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,176
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['mapred']
> > >
> > > 2017-04-25 14:12:50,223
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hbase']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,224
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hbase']
> > >
> > > 2017-04-25 14:12:50,271
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hcat']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,272
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hcat']
> > >
> > > 2017-04-25 14:12:50,320
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > >
> > > 2017-04-25 14:12:50,333
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
> exist
> > >
> > > 2017-04-25 14:12:50,334
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to 555
> > >
> > > 2017-04-25 14:12:50,335
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
> $(id
> > -u
> > > ambari-qa) -gt 1000) || (false)'}
> > >
> > > 2017-04-25 14:12:50,364
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > >
> > > 2017-04-25 14:12:50,365
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
> True,
> > > 'mode': 0775, 'cd_access': 'a'}
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> owner
> > > for /tmp/hbase-hbase from 0 to hbase
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /tmp/hbase-hbase from 755 to 775
> > >
> > > 2017-04-25 14:12:50,368
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > >
> > > 2017-04-25 14:12:50,369
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > >
/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>
> > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > >
> > > 2017-04-25 14:12:50,393
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > >
/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>
> > > due to not_if
> > >
> > > 2017-04-25 14:12:50,394
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Group['hdfs']
> {}
> > >
> > > 2017-04-25 14:12:50,395
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding group
> > > Group['hdfs']
> > >
> > > 2017-04-25 14:12:50,430
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > >
> > > 2017-04-25 14:12:50,431
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
> user
> > > hdfs
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/etc/hadoop'] {'mode': 0755}
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,475
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> {'owner':
> > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory
Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
>
> > > since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> owner
> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> group
> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hadoop
> > >
> > > 2017-04-25 14:12:50,477
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
> 755
> > > to 1777
> > >
> > > 2017-04-25 14:12:50,477
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > {'create_parents': True}
> > >
> > > 2017-04-25 14:12:50,478
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> since
> > > it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,479
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> {'content':
> > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > >
> > > 2017-04-25 14:12:50,504
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Downloading
> the
> > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > linux-x64.tar.gz
> > >
> > > 2017-04-25 14:12:53,793
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > Directory['/usr/jdk64'] {}
> > >
> > > 2017-04-25 14:12:53,793
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:53,794
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > Execute[('chmod',
> > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > >
> > > 2017-04-25 14:12:53,824
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute['cd
> > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz && ambari-sudo.sh
> cp
> > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > >
> > > 2017-04-25 14:13:00,439
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > ['delete']}
> > >
> > > 2017-04-25 14:13:00,440
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and
> all
> > > its content
> > >
> > > 2017-04-25 14:13:00,620
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access':
> 'a'}
> > >
> > > 2017-04-25 14:13:00,621
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Execute[('chmod',
> > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > >
> > > 2017-04-25 14:13:00,682
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Initializing
> 2
> > > repositories
> > >
> > > 2017-04-25 14:13:00,683
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> ['create'],
> > > 'components': [u'HDP', 'main'], 'repo_template':
> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > 'mirror_list': None}
> > >
> > > 2017-04-25 14:13:00,709
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
> > >
> > > 2017-04-25 14:13:00,711
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > >
> > > 2017-04-25 14:13:00,711
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://public-repo-1.
> > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> ['create'],
> > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS',
> > > 'mirror_list': None}
> > >
> > > 2017-04-25 14:13:00,717
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> InlineTemplate(...)}
> > >
> > > 2017-04-25 14:13:00,717
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > >
> > > 2017-04-25 14:13:00,718
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Package['unzip']
> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:00,846
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > >
> > > 2017-04-25 14:13:03,413
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> Package['curl']
> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:03,429
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > installation of existing package curl
> > >
> > > 2017-04-25 14:13:03,430
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:03,451
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
> > >
> > > 2017-04-25 14:13:04,948
> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES
> Data
> > > Node
> > >
> > > 2017-04-25 14:13:04,952
> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> False,
> > > 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:05,086
> > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > elasticsearch-2.3.3')
> > >
> > >
> > >
> > > Command failed after 1 tries
> > >
> > >
> > >
> > >
> > >
> > > On April 25, 2017 at 13:47:17, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > Yes, I’m going to run through again and get the logs. The basic issue
> > is,
> > > no repo tab -> if I should have one,
> > > and the ES yum command failing with nothing to do, which stops
> deployment
> > > on all the nodes.
> > >
> > >
> > >
> > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > michael.miklavcic@gmail.com) wrote:
> > >
> > > I've been doing an internal hwx deployment (HCP) recently for testing
> > > purposes. I have some ES tweaks to work out, but other than that it
> has
> > > been running smoothly. Can you share any log info or screenshots?
> > >
> > > Mike
> > >
> > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > > Should I see the repo tab in the ambari wizard for metron at this
> > point?
> > > > Asking for a friend who can’t get the wizard to work :)
> > > >
> > >
> > >
> >
>
>
Re: Ambari Wizard: Repo Tab
Posted by David Lyle <dl...@gmail.com>.
That would do it. It requires 2.4.2+. I would have sworn I put that in the
README, but I must have only annotated the PR. :(
I'll get that in asap.
-D...
On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ot...@gmail.com>
wrote:
> 2.4.1 as a matter of fact
>
>
> On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com) wrote:
>
> Any chance they're running Ambari < 2.4.2?
>
> -D...
>
>
> On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
> michael.miklavcic@gmail.com> wrote:
>
> > Hey Otto, I don't have wizard screenshots in front of me right now to
> say
> > for sure, but I do see a repoinfo.xml in the mpack. I haven't run into
> > anything like that, but next time I run through the install I can take
> > another look.
> >
> > https://github.com/apache/incubator-metron/blob/master/
> > metron-deployment/packaging/ambari/metron-mpack/src/main/
> > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
> >
> >
> > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ot...@gmail.com>
> > wrote:
> >
> > > stderr:
> > >
> > > Traceback (most recent call last):
> > >
> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> > >
> > > Elasticsearch().execute()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > libraries/script/script.py",
> > > line 280, in execute
> > >
> > > method(env)
> > >
> > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> > >
> > > self.install_packages(env)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > libraries/script/script.py",
> > > line 567, in install_packages
> > >
> > > retry_count=agent_stack_retry_count)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/base.py",
> > > line 155, in __init__
> > >
> > > self.env.run()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/environment.py",
> > > line 160, in run
> > >
> > > self.run_action(resource, action)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/environment.py",
> > > line 124, in run_action
> > >
> > > provider_action()
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 54, in action_install
> > >
> > > self.install_package(package_name, self.resource.use_repos,
> > > self.resource.skip_repos)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/yumrpm.py", line 49, in install_package
> > >
> > > self.checked_call_with_retries(cmd, sudo=True,
> > > logoutput=self.get_logoutput())
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 83, in
> > checked_call_with_retries
> > >
> > > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > > core/providers/package/__init__.py", line 91, in _call_with_retries
> > >
> > > code, out = func(cmd, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 71, in inner
> > >
> > > result = function(command, **kwargs)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 93, in checked_call
> > >
> > > tries=tries, try_sleep=try_sleep)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 141, in _call_wrapper
> > >
> > > result = _call(command, **kwargs_copy)
> > >
> > > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/shell.py",
> > > line 294, in _call
> > >
> > > raise Fail(err_msg)
> > >
> > > resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum
> -d
> > 0
> > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to do
> > >
> > > stdout:
> > >
> > > 2017-04-25 14:12:48,669
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using hadoop
> > conf
> > > dir: /usr/hdp/current/hadoop-client/conf
> > >
> > > 2017-04-25 14:12:48,671
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['metron']
> > {}
> > >
> > > 2017-04-25 14:12:48,673
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['metron']
> > >
> > > 2017-04-25 14:12:48,740
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['livy']
> {}
> > >
> > > 2017-04-25 14:12:48,741
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['livy']
> > >
> > > 2017-04-25 14:12:48,774
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > > Group['elasticsearch'] {}
> > >
> > > 2017-04-25 14:12:48,775
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['elasticsearch']
> > >
> > > 2017-04-25 14:12:48,812
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['spark']
> > {}
> > >
> > > 2017-04-25 14:12:48,812
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['spark']
> > >
> > > 2017-04-25 14:12:48,848
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['zeppelin']
> > > {}
> > >
> > > 2017-04-25 14:12:48,849
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['zeppelin']
> > >
> > > 2017-04-25 14:12:48,884
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['hadoop']
> > {}
> > >
> > > 2017-04-25 14:12:48,885
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['hadoop']
> > >
> > > 2017-04-25 14:12:48,921
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['kibana']
> > {}
> > >
> > > 2017-04-25 14:12:48,922
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > > Group['kibana']
> > >
> > > 2017-04-25 14:12:48,959
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['users']
> > {}
> > >
> > > 2017-04-25 14:12:48,960
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - User['hive']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:48,961
> > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding user
> > > User['hive']
> > >
> > > 2017-04-25 14:12:49,227
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['storm']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,227
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['storm']
> > >
> > > 2017-04-25 14:12:49,388
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['zookeeper']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,388
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['zookeeper']
> > >
> > > 2017-04-25 14:12:49,518
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > > 'groups': [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,518
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['infra-solr']
> > >
> > > 2017-04-25 14:12:49,578
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ams']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,578
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['ams']
> > >
> > > 2017-04-25 14:12:49,625
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['tez']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> > >
> > > 2017-04-25 14:12:49,626
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['tez']
> > >
> > > 2017-04-25 14:12:49,660
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['zeppelin']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,660
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['zeppelin']
> > >
> > > 2017-04-25 14:12:49,705
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['metron']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,705
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['metron']
> > >
> > > 2017-04-25 14:12:49,760
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['livy']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,760
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['livy']
> > >
> > > 2017-04-25 14:12:49,803
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > > 'groups': [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,804
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['elasticsearch']
> > >
> > > 2017-04-25 14:12:49,837
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['spark']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,837
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['spark']
> > >
> > > 2017-04-25 14:12:49,885
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['ambari-qa']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> > >
> > > 2017-04-25 14:12:49,885
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['ambari-qa']
> > >
> > > 2017-04-25 14:12:49,928
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['flume']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,928
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['flume']
> > >
> > > 2017-04-25 14:12:49,972
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['kafka']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:49,973
> > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > > User['kafka']
> > >
> > > 2017-04-25 14:12:50,025
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,026
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hdfs']
> > >
> > > 2017-04-25 14:12:50,075
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['yarn']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,075
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['yarn']
> > >
> > > 2017-04-25 14:12:50,125
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['kibana']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,126
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['kibana']
> > >
> > > 2017-04-25 14:12:50,175
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> User['mapred']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,176
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['mapred']
> > >
> > > 2017-04-25 14:12:50,223
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hbase']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,224
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hbase']
> > >
> > > 2017-04-25 14:12:50,271
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hcat']
> > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups':
> [u'hadoop']}
> > >
> > > 2017-04-25 14:12:50,272
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > > User['hcat']
> > >
> > > 2017-04-25 14:12:50,320
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > >
> > > 2017-04-25 14:12:50,333
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't
> exist
> > >
> > > 2017-04-25 14:12:50,334
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to 555
> > >
> > > 2017-04-25 14:12:50,335
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test
> $(id
> > -u
> > > ambari-qa) -gt 1000) || (false)'}
> > >
> > > 2017-04-25 14:12:50,364
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> > >
> > > 2017-04-25 14:12:50,365
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
> True,
> > > 'mode': 0775, 'cd_access': 'a'}
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> owner
> > > for /tmp/hbase-hbase from 0 to hbase
> > >
> > > 2017-04-25 14:12:50,366
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /tmp/hbase-hbase from 755 to 775
> > >
> > > 2017-04-25 14:12:50,368
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> > >
> > > 2017-04-25 14:12:50,369
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>
> > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> > >
> > > 2017-04-25 14:12:50,393
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
>
> > > due to not_if
> > >
> > > 2017-04-25 14:12:50,394
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Group['hdfs']
> {}
> > >
> > > 2017-04-25 14:12:50,395
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding group
> > > Group['hdfs']
> > >
> > > 2017-04-25 14:12:50,430
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> > >
> > > 2017-04-25 14:12:50,431
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying
> user
> > > hdfs
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/etc/hadoop'] {'mode': 0755}
> > >
> > > 2017-04-25 14:12:50,474
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/etc/hadoop'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,475
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> {'owner':
> > > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
>
> > > since it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> owner
> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
> > >
> > > 2017-04-25 14:12:50,476
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> group
> > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hadoop
> > >
> > > 2017-04-25 14:12:50,477
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from
> 755
> > > to 1777
> > >
> > > 2017-04-25 14:12:50,477
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > > {'create_parents': True}
> > >
> > > 2017-04-25 14:12:50,478
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> since
> > > it doesn't exist.
> > >
> > > 2017-04-25 14:12:50,479
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz']
> {'content':
> > > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> > >
> > > 2017-04-25 14:12:50,504
> > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Downloading
> the
> > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > > linux-x64.tar.gz
> > >
> > > 2017-04-25 14:12:53,793
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > > Directory['/usr/jdk64'] {}
> > >
> > > 2017-04-25 14:12:53,793
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > > directory Directory['/usr/jdk64'] since it doesn't exist.
> > >
> > > 2017-04-25 14:12:53,794
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > Execute[('chmod',
> > > 'a+x', u'/usr/jdk64')] {'sudo': True}
> > >
> > > 2017-04-25 14:12:53,824
> > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute['cd
> > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz && ambari-sudo.sh
> cp
> > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> > >
> > > 2017-04-25 14:13:00,439
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > > ['delete']}
> > >
> > > 2017-04-25 14:13:00,440
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and
> all
> > > its content
> > >
> > > 2017-04-25 14:13:00,620
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access':
> 'a'}
> > >
> > > 2017-04-25 14:13:00,621
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Execute[('chmod',
> > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> > >
> > > 2017-04-25 14:13:00,682
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Initializing
> 2
> > > repositories
> > >
> > > 2017-04-25 14:13:00,683
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action':
> ['create'],
> > > 'components': [u'HDP', 'main'], 'repo_template':
> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > > 'mirror_list': None}
> > >
> > > 2017-04-25 14:13:00,709
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
> > >
> > > 2017-04-25 14:13:00,711
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> > >
> > > 2017-04-25 14:13:00,711
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://public-repo-1.
> > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
> ['create'],
> > > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS',
> > > 'mirror_list': None}
> > >
> > > 2017-04-25 14:13:00,717
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
> InlineTemplate(...)}
> > >
> > > 2017-04-25 14:13:00,717
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> > >
> > > 2017-04-25 14:13:00,718
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Package['unzip']
> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:00,846
> > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> > >
> > > 2017-04-25 14:13:03,413
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> Package['curl']
> > > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:03,429
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > > installation of existing package curl
> > >
> > > 2017-04-25 14:13:03,430
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > > 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:03,451
> > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
> > >
> > > 2017-04-25 14:13:04,948
> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES
> Data
> > > Node
> > >
> > > 2017-04-25 14:13:04,952
> > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability':
> False,
> > > 'retry_count': 5}
> > >
> > > 2017-04-25 14:13:05,086
> > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > > elasticsearch-2.3.3')
> > >
> > >
> > >
> > > Command failed after 1 tries
> > >
> > >
> > >
> > >
> > >
> > > On April 25, 2017 at 13:47:17, Otto Fowler (ottobackwards@gmail.com)
> > > wrote:
> > >
> > > Yes, I’m going to run through again and get the logs. The basic issue
> > is,
> > > no repo tab -> if I should have one,
> > > and the ES yum command failing with nothing to do, which stops
> deployment
> > > on all the nodes.
> > >
> > >
> > >
> > > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > > michael.miklavcic@gmail.com) wrote:
> > >
> > > I've been doing an internal hwx deployment (HCP) recently for testing
> > > purposes. I have some ES tweaks to work out, but other than that it
> has
> > > been running smoothly. Can you share any log info or screenshots?
> > >
> > > Mike
> > >
> > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > > Should I see the repo tab in the ambari wizard for metron at this
> > point?
> > > > Asking for a friend who can’t get the wizard to work :)
> > > >
> > >
> > >
> >
>
>
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
2.4.1 as a matter of fact
On April 25, 2017 at 15:29:43, David Lyle (dlyle65535@gmail.com) wrote:
Any chance they're running Ambari < 2.4.2?
-D...
On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
michael.miklavcic@gmail.com> wrote:
> Hey Otto, I don't have wizard screenshots in front of me right now to say
> for sure, but I do see a repoinfo.xml in the mpack. I haven't run into
> anything like that, but next time I run through the install I can take
> another look.
>
> https://github.com/apache/incubator-metron/blob/master/
> metron-deployment/packaging/ambari/metron-mpack/src/main/
> resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
>
>
> On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > stderr:
> >
> > Traceback (most recent call last):
> >
> > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> >
> > Elasticsearch().execute()
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> libraries/script/script.py",
> > line 280, in execute
> >
> > method(env)
> >
> > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> >
> > self.install_packages(env)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> libraries/script/script.py",
> > line 567, in install_packages
> >
> > retry_count=agent_stack_retry_count)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> core/base.py",
> > line 155, in __init__
> >
> > self.env.run()
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> core/environment.py",
> > line 160, in run
> >
> > self.run_action(resource, action)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> core/environment.py",
> > line 124, in run_action
> >
> > provider_action()
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/providers/package/__init__.py", line 54, in action_install
> >
> > self.install_package(package_name, self.resource.use_repos,
> > self.resource.skip_repos)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/providers/package/yumrpm.py", line 49, in install_package
> >
> > self.checked_call_with_retries(cmd, sudo=True,
> > logoutput=self.get_logoutput())
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/providers/package/__init__.py", line 83, in
> checked_call_with_retries
> >
> > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/providers/package/__init__.py", line 91, in _call_with_retries
> >
> > code, out = func(cmd, **kwargs)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> core/shell.py",
> > line 71, in inner
> >
> > result = function(command, **kwargs)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> core/shell.py",
> > line 93, in checked_call
> >
> > tries=tries, try_sleep=try_sleep)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> core/shell.py",
> > line 141, in _call_wrapper
> >
> > result = _call(command, **kwargs_copy)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> core/shell.py",
> > line 294, in _call
> >
> > raise Fail(err_msg)
> >
> > resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum -d
> 0
> > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to do
> >
> > stdout:
> >
> > 2017-04-25 14:12:48,669
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using hadoop
> conf
> > dir: /usr/hdp/current/hadoop-client/conf
> >
> > 2017-04-25 14:12:48,671
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['metron']
> {}
> >
> > 2017-04-25 14:12:48,673
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > Group['metron']
> >
> > 2017-04-25 14:12:48,740
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['livy']
{}
> >
> > 2017-04-25 14:12:48,741
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > Group['livy']
> >
> > 2017-04-25 14:12:48,774
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['elasticsearch'] {}
> >
> > 2017-04-25 14:12:48,775
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > Group['elasticsearch']
> >
> > 2017-04-25 14:12:48,812
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['spark']
> {}
> >
> > 2017-04-25 14:12:48,812
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > Group['spark']
> >
> > 2017-04-25 14:12:48,848
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['zeppelin']
> > {}
> >
> > 2017-04-25 14:12:48,849
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > Group['zeppelin']
> >
> > 2017-04-25 14:12:48,884
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['hadoop']
> {}
> >
> > 2017-04-25 14:12:48,885
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > Group['hadoop']
> >
> > 2017-04-25 14:12:48,921
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['kibana']
> {}
> >
> > 2017-04-25 14:12:48,922
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > Group['kibana']
> >
> > 2017-04-25 14:12:48,959
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['users']
> {}
> >
> > 2017-04-25 14:12:48,960
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - User['hive']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:48,961
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding user
> > User['hive']
> >
> > 2017-04-25 14:12:49,227
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['storm']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,227
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['storm']
> >
> > 2017-04-25 14:12:49,388
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['zookeeper']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,388
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['zookeeper']
> >
> > 2017-04-25 14:12:49,518
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,518
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['infra-solr']
> >
> > 2017-04-25 14:12:49,578
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ams']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,578
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['ams']
> >
> > 2017-04-25 14:12:49,625
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['tez']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> >
> > 2017-04-25 14:12:49,626
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['tez']
> >
> > 2017-04-25 14:12:49,660
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
User['zeppelin']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,660
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['zeppelin']
> >
> > 2017-04-25 14:12:49,705
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['metron']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,705
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['metron']
> >
> > 2017-04-25 14:12:49,760
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['livy']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,760
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['livy']
> >
> > 2017-04-25 14:12:49,803
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,804
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['elasticsearch']
> >
> > 2017-04-25 14:12:49,837
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['spark']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,837
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['spark']
> >
> > 2017-04-25 14:12:49,885
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['ambari-qa']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> >
> > 2017-04-25 14:12:49,885
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['ambari-qa']
> >
> > 2017-04-25 14:12:49,928
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['flume']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,928
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['flume']
> >
> > 2017-04-25 14:12:49,972
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['kafka']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,973
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['kafka']
> >
> > 2017-04-25 14:12:50,025
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:50,026
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > User['hdfs']
> >
> > 2017-04-25 14:12:50,075
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['yarn']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:50,075
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > User['yarn']
> >
> > 2017-04-25 14:12:50,125
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['kibana']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:50,126
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > User['kibana']
> >
> > 2017-04-25 14:12:50,175
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['mapred']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:50,176
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > User['mapred']
> >
> > 2017-04-25 14:12:50,223
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hbase']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:50,224
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > User['hbase']
> >
> > 2017-04-25 14:12:50,271
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hcat']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:50,272
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > User['hcat']
> >
> > 2017-04-25 14:12:50,320
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> >
> > 2017-04-25 14:12:50,333
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't exist
> >
> > 2017-04-25 14:12:50,334
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to 555
> >
> > 2017-04-25 14:12:50,335
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id
> -u
> > ambari-qa) -gt 1000) || (false)'}
> >
> > 2017-04-25 14:12:50,364
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> >
> > 2017-04-25 14:12:50,365
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents':
True,
> > 'mode': 0775, 'cd_access': 'a'}
> >
> > 2017-04-25 14:12:50,366
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> >
> > 2017-04-25 14:12:50,366
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing owner
> > for /tmp/hbase-hbase from 0 to hbase
> >
> > 2017-04-25 14:12:50,366
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > permission for /tmp/hbase-hbase from 755 to 775
> >
> > 2017-04-25 14:12:50,368
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> >
> > 2017-04-25 14:12:50,369
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> >
> > 2017-04-25 14:12:50,393
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> > due to not_if
> >
> > 2017-04-25 14:12:50,394
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Group['hdfs']
{}
> >
> > 2017-04-25 14:12:50,395
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding group
> > Group['hdfs']
> >
> > 2017-04-25 14:12:50,430
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> >
> > 2017-04-25 14:12:50,431
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying user
> > hdfs
> >
> > 2017-04-25 14:12:50,474
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> >
> > 2017-04-25 14:12:50,474
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Directory['/etc/hadoop'] {'mode': 0755}
> >
> > 2017-04-25 14:12:50,474
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > directory Directory['/etc/hadoop'] since it doesn't exist.
> >
> > 2017-04-25 14:12:50,475
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner':
> > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> >
> > 2017-04-25 14:12:50,476
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > since it doesn't exist.
> >
> > 2017-04-25 14:12:50,476
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing owner
> > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
> >
> > 2017-04-25 14:12:50,476
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing group
> > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hadoop
> >
> > 2017-04-25 14:12:50,477
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 755
> > to 1777
> >
> > 2017-04-25 14:12:50,477
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > {'create_parents': True}
> >
> > 2017-04-25 14:12:50,478
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
since
> > it doesn't exist.
> >
> > 2017-04-25 14:12:50,479
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'] {'content':
> > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> >
> > 2017-04-25 14:12:50,504
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Downloading the
> > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > linux-x64.tar.gz
> >
> > 2017-04-25 14:12:53,793
> > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > Directory['/usr/jdk64'] {}
> >
> > 2017-04-25 14:12:53,793
> > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > directory Directory['/usr/jdk64'] since it doesn't exist.
> >
> > 2017-04-25 14:12:53,794
> > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> Execute[('chmod',
> > 'a+x', u'/usr/jdk64')] {'sudo': True}
> >
> > 2017-04-25 14:12:53,824
> > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute['cd
> > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz && ambari-sudo.sh
cp
> > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> >
> > 2017-04-25 14:13:00,439
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > ['delete']}
> >
> > 2017-04-25 14:13:00,440
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and all
> > its content
> >
> > 2017-04-25 14:13:00,620
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access':
'a'}
> >
> > 2017-04-25 14:13:00,621
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Execute[('chmod',
> > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> >
> > 2017-04-25 14:13:00,682
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Initializing 2
> > repositories
> >
> > 2017-04-25 14:13:00,683
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action': ['create'],
> > 'components': [u'HDP', 'main'], 'repo_template':
> > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > 'mirror_list': None}
> >
> > 2017-04-25 14:13:00,709
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
> >
> > 2017-04-25 14:13:00,711
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> >
> > 2017-04-25 14:13:00,711
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://public-repo-1.
> > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action':
['create'],
> > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS',
> > 'mirror_list': None}
> >
> > 2017-04-25 14:13:00,717
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content':
InlineTemplate(...)}
> >
> > 2017-04-25 14:13:00,717
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> >
> > 2017-04-25 14:13:00,718
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
Package['unzip']
> > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> >
> > 2017-04-25 14:13:00,846
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> >
> > 2017-04-25 14:13:03,413
> > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Package['curl']
> > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> >
> > 2017-04-25 14:13:03,429
> > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > installation of existing package curl
> >
> > 2017-04-25 14:13:03,430
> > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > 'retry_count': 5}
> >
> > 2017-04-25 14:13:03,451
> > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
> >
> > 2017-04-25 14:13:04,948
> > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES Data
> > Node
> >
> > 2017-04-25 14:13:04,952
> > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability': False,
> > 'retry_count': 5}
> >
> > 2017-04-25 14:13:05,086
> > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > elasticsearch-2.3.3')
> >
> >
> >
> > Command failed after 1 tries
> >
> >
> >
> >
> >
> > On April 25, 2017 at 13:47:17, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Yes, I’m going to run through again and get the logs. The basic issue
> is,
> > no repo tab -> if I should have one,
> > and the ES yum command failing with nothing to do, which stops
deployment
> > on all the nodes.
> >
> >
> >
> > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > michael.miklavcic@gmail.com) wrote:
> >
> > I've been doing an internal hwx deployment (HCP) recently for testing
> > purposes. I have some ES tweaks to work out, but other than that it has
> > been running smoothly. Can you share any log info or screenshots?
> >
> > Mike
> >
> > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com>
wrote:
> >
> > > Hi,
> > > Should I see the repo tab in the ambari wizard for metron at this
> point?
> > > Asking for a friend who can’t get the wizard to work :)
> > >
> >
> >
>
Re: Ambari Wizard: Repo Tab
Posted by David Lyle <dl...@gmail.com>.
Any chance they're running Ambari < 2.4.2?
-D...
On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic <
michael.miklavcic@gmail.com> wrote:
> Hey Otto, I don't have wizard screenshots in front of me right now to say
> for sure, but I do see a repoinfo.xml in the mpack. I haven't run into
> anything like that, but next time I run through the install I can take
> another look.
>
> https://github.com/apache/incubator-metron/blob/master/
> metron-deployment/packaging/ambari/metron-mpack/src/main/
> resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
>
>
> On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ot...@gmail.com>
> wrote:
>
> > stderr:
> >
> > Traceback (most recent call last):
> >
> > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
> >
> > Elasticsearch().execute()
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> libraries/script/script.py",
> > line 280, in execute
> >
> > method(env)
> >
> > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> > 2.3.3/package/scripts/elastic_slave.py", line 32, in install
> >
> > self.install_packages(env)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> libraries/script/script.py",
> > line 567, in install_packages
> >
> > retry_count=agent_stack_retry_count)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> core/base.py",
> > line 155, in __init__
> >
> > self.env.run()
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> core/environment.py",
> > line 160, in run
> >
> > self.run_action(resource, action)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> core/environment.py",
> > line 124, in run_action
> >
> > provider_action()
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/providers/package/__init__.py", line 54, in action_install
> >
> > self.install_package(package_name, self.resource.use_repos,
> > self.resource.skip_repos)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/providers/package/yumrpm.py", line 49, in install_package
> >
> > self.checked_call_with_retries(cmd, sudo=True,
> > logoutput=self.get_logoutput())
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/providers/package/__init__.py", line 83, in
> checked_call_with_retries
> >
> > return self._call_with_retries(cmd, is_checked=True, **kwargs)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> > core/providers/package/__init__.py", line 91, in _call_with_retries
> >
> > code, out = func(cmd, **kwargs)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> core/shell.py",
> > line 71, in inner
> >
> > result = function(command, **kwargs)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> core/shell.py",
> > line 93, in checked_call
> >
> > tries=tries, try_sleep=try_sleep)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> core/shell.py",
> > line 141, in _call_wrapper
> >
> > result = _call(command, **kwargs_copy)
> >
> > File "/usr/lib/python2.6/site-packages/resource_management/
> core/shell.py",
> > line 294, in _call
> >
> > raise Fail(err_msg)
> >
> > resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum -d
> 0
> > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to do
> >
> > stdout:
> >
> > 2017-04-25 14:12:48,669
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using hadoop
> conf
> > dir: /usr/hdp/current/hadoop-client/conf
> >
> > 2017-04-25 14:12:48,671
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['metron']
> {}
> >
> > 2017-04-25 14:12:48,673
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > Group['metron']
> >
> > 2017-04-25 14:12:48,740
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['livy'] {}
> >
> > 2017-04-25 14:12:48,741
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > Group['livy']
> >
> > 2017-04-25 14:12:48,774
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> > Group['elasticsearch'] {}
> >
> > 2017-04-25 14:12:48,775
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > Group['elasticsearch']
> >
> > 2017-04-25 14:12:48,812
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['spark']
> {}
> >
> > 2017-04-25 14:12:48,812
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > Group['spark']
> >
> > 2017-04-25 14:12:48,848
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['zeppelin']
> > {}
> >
> > 2017-04-25 14:12:48,849
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > Group['zeppelin']
> >
> > 2017-04-25 14:12:48,884
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['hadoop']
> {}
> >
> > 2017-04-25 14:12:48,885
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > Group['hadoop']
> >
> > 2017-04-25 14:12:48,921
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['kibana']
> {}
> >
> > 2017-04-25 14:12:48,922
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> > Group['kibana']
> >
> > 2017-04-25 14:12:48,959
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['users']
> {}
> >
> > 2017-04-25 14:12:48,960
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - User['hive']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:48,961
> > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding user
> > User['hive']
> >
> > 2017-04-25 14:12:49,227
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['storm']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,227
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['storm']
> >
> > 2017-04-25 14:12:49,388
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['zookeeper']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,388
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['zookeeper']
> >
> > 2017-04-25 14:12:49,518
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,518
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['infra-solr']
> >
> > 2017-04-25 14:12:49,578
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ams']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,578
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['ams']
> >
> > 2017-04-25 14:12:49,625
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['tez']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> >
> > 2017-04-25 14:12:49,626
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['tez']
> >
> > 2017-04-25 14:12:49,660
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['zeppelin']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,660
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['zeppelin']
> >
> > 2017-04-25 14:12:49,705
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['metron']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,705
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['metron']
> >
> > 2017-04-25 14:12:49,760
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['livy']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,760
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['livy']
> >
> > 2017-04-25 14:12:49,803
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> > 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,804
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['elasticsearch']
> >
> > 2017-04-25 14:12:49,837
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['spark']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,837
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['spark']
> >
> > 2017-04-25 14:12:49,885
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['ambari-qa']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> >
> > 2017-04-25 14:12:49,885
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['ambari-qa']
> >
> > 2017-04-25 14:12:49,928
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['flume']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,928
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['flume']
> >
> > 2017-04-25 14:12:49,972
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['kafka']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:49,973
> > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> > User['kafka']
> >
> > 2017-04-25 14:12:50,025
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:50,026
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > User['hdfs']
> >
> > 2017-04-25 14:12:50,075
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['yarn']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:50,075
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > User['yarn']
> >
> > 2017-04-25 14:12:50,125
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['kibana']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:50,126
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > User['kibana']
> >
> > 2017-04-25 14:12:50,175
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['mapred']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:50,176
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > User['mapred']
> >
> > 2017-04-25 14:12:50,223
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hbase']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:50,224
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > User['hbase']
> >
> > 2017-04-25 14:12:50,271
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hcat']
> > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> >
> > 2017-04-25 14:12:50,272
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> > User['hcat']
> >
> > 2017-04-25 14:12:50,320
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> >
> > 2017-04-25 14:12:50,333
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't exist
> >
> > 2017-04-25 14:12:50,334
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to 555
> >
> > 2017-04-25 14:12:50,335
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id
> -u
> > ambari-qa) -gt 1000) || (false)'}
> >
> > 2017-04-25 14:12:50,364
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
> >
> > 2017-04-25 14:12:50,365
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents': True,
> > 'mode': 0775, 'cd_access': 'a'}
> >
> > 2017-04-25 14:12:50,366
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
> >
> > 2017-04-25 14:12:50,366
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing owner
> > for /tmp/hbase-hbase from 0 to hbase
> >
> > 2017-04-25 14:12:50,366
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > permission for /tmp/hbase-hbase from 755 to 775
> >
> > 2017-04-25 14:12:50,368
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> > StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> >
> > 2017-04-25 14:12:50,369
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> >
> > 2017-04-25 14:12:50,393
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> > due to not_if
> >
> > 2017-04-25 14:12:50,394
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Group['hdfs'] {}
> >
> > 2017-04-25 14:12:50,395
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding group
> > Group['hdfs']
> >
> > 2017-04-25 14:12:50,430
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
> >
> > 2017-04-25 14:12:50,431
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying user
> > hdfs
> >
> > 2017-04-25 14:12:50,474
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
> >
> > 2017-04-25 14:12:50,474
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Directory['/etc/hadoop'] {'mode': 0755}
> >
> > 2017-04-25 14:12:50,474
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > directory Directory['/etc/hadoop'] since it doesn't exist.
> >
> > 2017-04-25 14:12:50,475
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner':
> > 'hdfs', 'group': 'hadoop', 'mode': 01777}
> >
> > 2017-04-25 14:12:50,476
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> > since it doesn't exist.
> >
> > 2017-04-25 14:12:50,476
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing owner
> > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
> >
> > 2017-04-25 14:12:50,476
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing group
> > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hadoop
> >
> > 2017-04-25 14:12:50,477
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 755
> > to 1777
> >
> > 2017-04-25 14:12:50,477
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> > {'create_parents': True}
> >
> > 2017-04-25 14:12:50,478
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/'] since
> > it doesn't exist.
> >
> > 2017-04-25 14:12:50,479
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'] {'content':
> > DownloadSource('http://css-fx-met-poc-82:8080/resources//
> > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
> >
> > 2017-04-25 14:12:50,504
> > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Downloading the
> > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> > linux-x64.tar.gz
> >
> > 2017-04-25 14:12:53,793
> > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> > Directory['/usr/jdk64'] {}
> >
> > 2017-04-25 14:12:53,793
> > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> > directory Directory['/usr/jdk64'] since it doesn't exist.
> >
> > 2017-04-25 14:12:53,794
> > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> Execute[('chmod',
> > 'a+x', u'/usr/jdk64')] {'sudo': True}
> >
> > 2017-04-25 14:12:53,824
> > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute['cd
> > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz && ambari-sudo.sh cp
> > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
> >
> > 2017-04-25 14:13:00,439
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> > ['delete']}
> >
> > 2017-04-25 14:13:00,440
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and all
> > its content
> >
> > 2017-04-25 14:13:00,620
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access': 'a'}
> >
> > 2017-04-25 14:13:00,621
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Execute[('chmod',
> > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
> >
> > 2017-04-25 14:13:00,682
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Initializing 2
> > repositories
> >
> > 2017-04-25 14:13:00,683
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action': ['create'],
> > 'components': [u'HDP', 'main'], 'repo_template':
> > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> > 'mirror_list': None}
> >
> > 2017-04-25 14:13:00,709
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
> >
> > 2017-04-25 14:13:00,711
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
> >
> > 2017-04-25 14:13:00,711
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://public-repo-1.
> > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action': ['create'],
> > 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS',
> > 'mirror_list': None}
> >
> > 2017-04-25 14:13:00,717
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content': InlineTemplate(...)}
> >
> > 2017-04-25 14:13:00,717
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
> >
> > 2017-04-25 14:13:00,718
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Package['unzip']
> > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> >
> > 2017-04-25 14:13:00,846
> > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
> >
> > 2017-04-25 14:13:03,413
> > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Package['curl']
> > {'retry_on_repo_unavailability': False, 'retry_count': 5}
> >
> > 2017-04-25 14:13:03,429
> > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> > installation of existing package curl
> >
> > 2017-04-25 14:13:03,430
> > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> > Package['hdp-select'] {'retry_on_repo_unavailability': False,
> > 'retry_count': 5}
> >
> > 2017-04-25 14:13:03,451
> > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
> >
> > 2017-04-25 14:13:04,948
> > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES Data
> > Node
> >
> > 2017-04-25 14:13:04,952
> > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability': False,
> > 'retry_count': 5}
> >
> > 2017-04-25 14:13:05,086
> > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> > elasticsearch-2.3.3')
> >
> >
> >
> > Command failed after 1 tries
> >
> >
> >
> >
> >
> > On April 25, 2017 at 13:47:17, Otto Fowler (ottobackwards@gmail.com)
> > wrote:
> >
> > Yes, I’m going to run through again and get the logs. The basic issue
> is,
> > no repo tab -> if I should have one,
> > and the ES yum command failing with nothing to do, which stops deployment
> > on all the nodes.
> >
> >
> >
> > On April 25, 2017 at 13:36:25, Michael Miklavcic (
> > michael.miklavcic@gmail.com) wrote:
> >
> > I've been doing an internal hwx deployment (HCP) recently for testing
> > purposes. I have some ES tweaks to work out, but other than that it has
> > been running smoothly. Can you share any log info or screenshots?
> >
> > Mike
> >
> > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com> wrote:
> >
> > > Hi,
> > > Should I see the repo tab in the ambari wizard for metron at this
> point?
> > > Asking for a friend who can’t get the wizard to work :)
> > >
> >
> >
>
Re: Ambari Wizard: Repo Tab
Posted by Michael Miklavcic <mi...@gmail.com>.
Hey Otto, I don't have wizard screenshots in front of me right now to say
for sure, but I do see a repoinfo.xml in the mpack. I haven't run into
anything like that, but next time I run through the install I can take
another look.
https://github.com/apache/incubator-metron/blob/master/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml
On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ot...@gmail.com>
wrote:
> stderr:
>
> Traceback (most recent call last):
>
> File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> 2.3.3/package/scripts/elastic_slave.py", line 71, in <module>
>
> Elasticsearch().execute()
>
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 280, in execute
>
> method(env)
>
> File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/
> 2.3.3/package/scripts/elastic_slave.py", line 32, in install
>
> self.install_packages(env)
>
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 567, in install_packages
>
> retry_count=agent_stack_retry_count)
>
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py",
> line 155, in __init__
>
> self.env.run()
>
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
> line 160, in run
>
> self.run_action(resource, action)
>
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
> line 124, in run_action
>
> provider_action()
>
> File "/usr/lib/python2.6/site-packages/resource_management/
> core/providers/package/__init__.py", line 54, in action_install
>
> self.install_package(package_name, self.resource.use_repos,
> self.resource.skip_repos)
>
> File "/usr/lib/python2.6/site-packages/resource_management/
> core/providers/package/yumrpm.py", line 49, in install_package
>
> self.checked_call_with_retries(cmd, sudo=True,
> logoutput=self.get_logoutput())
>
> File "/usr/lib/python2.6/site-packages/resource_management/
> core/providers/package/__init__.py", line 83, in checked_call_with_retries
>
> return self._call_with_retries(cmd, is_checked=True, **kwargs)
>
> File "/usr/lib/python2.6/site-packages/resource_management/
> core/providers/package/__init__.py", line 91, in _call_with_retries
>
> code, out = func(cmd, **kwargs)
>
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 71, in inner
>
> result = function(command, **kwargs)
>
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 93, in checked_call
>
> tries=tries, try_sleep=try_sleep)
>
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 141, in _call_wrapper
>
> result = _call(command, **kwargs_copy)
>
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 294, in _call
>
> raise Fail(err_msg)
>
> resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum -d 0
> -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to do
>
> stdout:
>
> 2017-04-25 14:12:48,669
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using hadoop conf
> dir: /usr/hdp/current/hadoop-client/conf
>
> 2017-04-25 14:12:48,671
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['metron'] {}
>
> 2017-04-25 14:12:48,673
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> Group['metron']
>
> 2017-04-25 14:12:48,740
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['livy'] {}
>
> 2017-04-25 14:12:48,741
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> Group['livy']
>
> 2017-04-25 14:12:48,774
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
> Group['elasticsearch'] {}
>
> 2017-04-25 14:12:48,775
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> Group['elasticsearch']
>
> 2017-04-25 14:12:48,812
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['spark'] {}
>
> 2017-04-25 14:12:48,812
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> Group['spark']
>
> 2017-04-25 14:12:48,848
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['zeppelin']
> {}
>
> 2017-04-25 14:12:48,849
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> Group['zeppelin']
>
> 2017-04-25 14:12:48,884
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['hadoop'] {}
>
> 2017-04-25 14:12:48,885
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> Group['hadoop']
>
> 2017-04-25 14:12:48,921
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['kibana'] {}
>
> 2017-04-25 14:12:48,922
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
> Group['kibana']
>
> 2017-04-25 14:12:48,959
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['users'] {}
>
> 2017-04-25 14:12:48,960
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - User['hive']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:48,961
> <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding user
> User['hive']
>
> 2017-04-25 14:12:49,227
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['storm']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:49,227
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> User['storm']
>
> 2017-04-25 14:12:49,388
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['zookeeper']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:49,388
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> User['zookeeper']
>
> 2017-04-25 14:12:49,518
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:49,518
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> User['infra-solr']
>
> 2017-04-25 14:12:49,578
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ams']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:49,578
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> User['ams']
>
> 2017-04-25 14:12:49,625
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['tez']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
>
> 2017-04-25 14:12:49,626
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> User['tez']
>
> 2017-04-25 14:12:49,660
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['zeppelin']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:49,660
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> User['zeppelin']
>
> 2017-04-25 14:12:49,705
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['metron']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:49,705
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> User['metron']
>
> 2017-04-25 14:12:49,760
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['livy']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:49,760
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> User['livy']
>
> 2017-04-25 14:12:49,803
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
> User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
> 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:49,804
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> User['elasticsearch']
>
> 2017-04-25 14:12:49,837
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['spark']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:49,837
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> User['spark']
>
> 2017-04-25 14:12:49,885
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ambari-qa']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
>
> 2017-04-25 14:12:49,885
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> User['ambari-qa']
>
> 2017-04-25 14:12:49,928
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['flume']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:49,928
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> User['flume']
>
> 2017-04-25 14:12:49,972
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['kafka']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:49,973
> <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
> User['kafka']
>
> 2017-04-25 14:12:50,025
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:50,026
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> User['hdfs']
>
> 2017-04-25 14:12:50,075
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['yarn']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:50,075
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> User['yarn']
>
> 2017-04-25 14:12:50,125
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['kibana']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:50,126
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> User['kibana']
>
> 2017-04-25 14:12:50,175
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['mapred']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:50,176
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> User['mapred']
>
> 2017-04-25 14:12:50,223
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hbase']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:50,224
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> User['hbase']
>
> 2017-04-25 14:12:50,271
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hcat']
> {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
>
> 2017-04-25 14:12:50,272
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
> User['hcat']
>
> 2017-04-25 14:12:50,320
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> StaticFile('changeToSecureUid.sh'), 'mode': 0555}
>
> 2017-04-25 14:12:50,333
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
> File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't exist
>
> 2017-04-25 14:12:50,334
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to 555
>
> 2017-04-25 14:12:50,335
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u
> ambari-qa) -gt 1000) || (false)'}
>
> 2017-04-25 14:12:50,364
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/
> ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
>
> 2017-04-25 14:12:50,365
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents': True,
> 'mode': 0775, 'cd_access': 'a'}
>
> 2017-04-25 14:12:50,366
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> directory Directory['/tmp/hbase-hbase'] since it doesn't exist.
>
> 2017-04-25 14:12:50,366
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing owner
> for /tmp/hbase-hbase from 0 to hbase
>
> 2017-04-25 14:12:50,366
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> permission for /tmp/hbase-hbase from 755 to 775
>
> 2017-04-25 14:12:50,368
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
> StaticFile('changeToSecureUid.sh'), 'mode': 0555}
>
> 2017-04-25 14:12:50,369
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
>
> 2017-04-25 14:12:50,393
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
> Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> due to not_if
>
> 2017-04-25 14:12:50,394
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Group['hdfs'] {}
>
> 2017-04-25 14:12:50,395
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding group
> Group['hdfs']
>
> 2017-04-25 14:12:50,430
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
> {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
>
> 2017-04-25 14:12:50,431
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying user
> hdfs
>
> 2017-04-25 14:12:50,474
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
>
> 2017-04-25 14:12:50,474
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Directory['/etc/hadoop'] {'mode': 0755}
>
> 2017-04-25 14:12:50,474
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> directory Directory['/etc/hadoop'] since it doesn't exist.
>
> 2017-04-25 14:12:50,475
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner':
> 'hdfs', 'group': 'hadoop', 'mode': 01777}
>
> 2017-04-25 14:12:50,476
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir']
> since it doesn't exist.
>
> 2017-04-25 14:12:50,476
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing owner
> for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
>
> 2017-04-25 14:12:50,476
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing group
> for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hadoop
>
> 2017-04-25 14:12:50,477
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing
> permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 755
> to 1777
>
> 2017-04-25 14:12:50,477
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/']
> {'create_parents': True}
>
> 2017-04-25 14:12:50,478
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating
> directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/'] since
> it doesn't exist.
>
> 2017-04-25 14:12:50,479
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
> File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'] {'content':
> DownloadSource('http://css-fx-met-poc-82:8080/resources//
> jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f
> /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
>
> 2017-04-25 14:12:50,504
> <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Downloading the
> file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-
> linux-x64.tar.gz
>
> 2017-04-25 14:12:53,793
> <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
> Directory['/usr/jdk64'] {}
>
> 2017-04-25 14:12:53,793
> <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating
> directory Directory['/usr/jdk64'] since it doesn't exist.
>
> 2017-04-25 14:12:53,794
> <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute[('chmod',
> 'a+x', u'/usr/jdk64')] {'sudo': True}
>
> 2017-04-25 14:12:53,824
> <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute['cd
> /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
> /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz && ambari-sudo.sh cp
> -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
>
> 2017-04-25 14:13:00,439
> <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action':
> ['delete']}
>
> 2017-04-25 14:13:00,440
> <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing
> directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and all
> its content
>
> 2017-04-25 14:13:00,620
> <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access': 'a'}
>
> 2017-04-25 14:13:00,621
> <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Execute[('chmod',
> '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
>
> 2017-04-25 14:13:00,682
> <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Initializing 2
> repositories
>
> 2017-04-25 14:13:00,683
> <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.
> hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action': ['create'],
> 'components': [u'HDP', 'main'], 'repo_template':
> '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
> 'mirror_list': None}
>
> 2017-04-25 14:13:00,709
> <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
>
> 2017-04-25 14:13:00,711
> <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
>
> 2017-04-25 14:13:00,711
> <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://public-repo-1.
> hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action': ['create'],
> 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS',
> 'mirror_list': None}
>
> 2017-04-25 14:13:00,717
> <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
> File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content': InlineTemplate(...)}
>
> 2017-04-25 14:13:00,717
> <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
> File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
>
> 2017-04-25 14:13:00,718
> <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Package['unzip']
> {'retry_on_repo_unavailability': False, 'retry_count': 5}
>
> 2017-04-25 14:13:00,846
> <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing
> package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
>
> 2017-04-25 14:13:03,413
> <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Package['curl']
> {'retry_on_repo_unavailability': False, 'retry_count': 5}
>
> 2017-04-25 14:13:03,429
> <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
> installation of existing package curl
>
> 2017-04-25 14:13:03,430
> <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
> Package['hdp-select'] {'retry_on_repo_unavailability': False,
> 'retry_count': 5}
>
> 2017-04-25 14:13:03,451
> <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing
> package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
>
> 2017-04-25 14:13:04,948
> <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES Data
> Node
>
> 2017-04-25 14:13:04,952
> <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
> Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability': False,
> 'retry_count': 5}
>
> 2017-04-25 14:13:05,086
> <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing
> package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
> elasticsearch-2.3.3')
>
>
>
> Command failed after 1 tries
>
>
>
>
>
> On April 25, 2017 at 13:47:17, Otto Fowler (ottobackwards@gmail.com)
> wrote:
>
> Yes, I’m going to run through again and get the logs. The basic issue is,
> no repo tab -> if I should have one,
> and the ES yum command failing with nothing to do, which stops deployment
> on all the nodes.
>
>
>
> On April 25, 2017 at 13:36:25, Michael Miklavcic (
> michael.miklavcic@gmail.com) wrote:
>
> I've been doing an internal hwx deployment (HCP) recently for testing
> purposes. I have some ES tweaks to work out, but other than that it has
> been running smoothly. Can you share any log info or screenshots?
>
> Mike
>
> On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com> wrote:
>
> > Hi,
> > Should I see the repo tab in the ambari wizard for metron at this point?
> > Asking for a friend who can’t get the wizard to work :)
> >
>
>
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
stderr:
Traceback (most recent call last):
File
"/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/2.3.3/package/scripts/elastic_slave.py",
line 71, in <module>
Elasticsearch().execute()
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 280, in execute
method(env)
File
"/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/2.3.3/package/scripts/elastic_slave.py",
line 32, in install
self.install_packages(env)
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 567, in install_packages
retry_count=agent_stack_retry_count)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py",
line 155, in __init__
self.env.run()
File
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
line 160, in run
self.run_action(resource, action)
File
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
line 124, in run_action
provider_action()
File
"/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py",
line 54, in action_install
self.install_package(package_name, self.resource.use_repos,
self.resource.skip_repos)
File
"/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py",
line 49, in install_package
self.checked_call_with_retries(cmd, sudo=True,
logoutput=self.get_logoutput())
File
"/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py",
line 83, in checked_call_with_retries
return self._call_with_retries(cmd, is_checked=True, **kwargs)
File
"/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py",
line 91, in _call_with_retries
code, out = func(cmd, **kwargs)
File
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line
71, in inner
result = function(command, **kwargs)
File
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line
93, in checked_call
tries=tries, try_sleep=try_sleep)
File
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line
141, in _call_wrapper
result = _call(command, **kwargs_copy)
File
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line
294, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum -d 0
-e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to do
stdout:
2017-04-25 14:12:48,669
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using hadoop conf
dir: /usr/hdp/current/hadoop-client/conf
2017-04-25 14:12:48,671
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['metron'] {}
2017-04-25 14:12:48,673
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
Group['metron']
2017-04-25 14:12:48,740
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['livy'] {}
2017-04-25 14:12:48,741
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
Group['livy']
2017-04-25 14:12:48,774
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> -
Group['elasticsearch'] {}
2017-04-25 14:12:48,775
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
Group['elasticsearch']
2017-04-25 14:12:48,812
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['spark'] {}
2017-04-25 14:12:48,812
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
Group['spark']
2017-04-25 14:12:48,848
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['zeppelin'] {}
2017-04-25 14:12:48,849
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
Group['zeppelin']
2017-04-25 14:12:48,884
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['hadoop'] {}
2017-04-25 14:12:48,885
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
Group['hadoop']
2017-04-25 14:12:48,921
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['kibana'] {}
2017-04-25 14:12:48,922
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding group
Group['kibana']
2017-04-25 14:12:48,959
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Group['users'] {}
2017-04-25 14:12:48,960
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - User['hive']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:48,961
<http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding user
User['hive']
2017-04-25 14:12:49,227
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['storm']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:49,227
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
User['storm']
2017-04-25 14:12:49,388
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['zookeeper']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:49,388
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
User['zookeeper']
2017-04-25 14:12:49,518
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['infra-solr']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:49,518
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
User['infra-solr']
2017-04-25 14:12:49,578
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ams'] {'gid':
'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:49,578
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
User['ams']
2017-04-25 14:12:49,625
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['tez'] {'gid':
'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
2017-04-25 14:12:49,626
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
User['tez']
2017-04-25 14:12:49,660
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['zeppelin']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:49,660
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
User['zeppelin']
2017-04-25 14:12:49,705
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['metron']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:49,705
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
User['metron']
2017-04-25 14:12:49,760
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['livy']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:49,760
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
User['livy']
2017-04-25 14:12:49,803
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> -
User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True,
'groups': [u'hadoop']}
2017-04-25 14:12:49,804
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
User['elasticsearch']
2017-04-25 14:12:49,837
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['spark']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:49,837
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
User['spark']
2017-04-25 14:12:49,885
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['ambari-qa']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
2017-04-25 14:12:49,885
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
User['ambari-qa']
2017-04-25 14:12:49,928
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['flume']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:49,928
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
User['flume']
2017-04-25 14:12:49,972
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - User['kafka']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:49,973
<http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding user
User['kafka']
2017-04-25 14:12:50,025
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:50,026
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
User['hdfs']
2017-04-25 14:12:50,075
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['yarn']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:50,075
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
User['yarn']
2017-04-25 14:12:50,125
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['kibana']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:50,126
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
User['kibana']
2017-04-25 14:12:50,175
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['mapred']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:50,176
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
User['mapred']
2017-04-25 14:12:50,223
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hbase']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:50,224
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
User['hbase']
2017-04-25 14:12:50,271
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hcat']
{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-04-25 14:12:50,272
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding user
User['hcat']
2017-04-25 14:12:50,320
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2017-04-25 14:12:50,333
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing
File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't exist
2017-04-25 14:12:50,334
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing permission
for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to 555
2017-04-25 14:12:50,335
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
/tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa']
{'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2017-04-25 14:12:50,364
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
/tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa']
due to not_if
2017-04-25 14:12:50,365
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents': True,
'mode': 0775, 'cd_access': 'a'}
2017-04-25 14:12:50,366
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating directory
Directory['/tmp/hbase-hbase'] since it doesn't exist.
2017-04-25 14:12:50,366
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing owner for
/tmp/hbase-hbase from 0 to hbase
2017-04-25 14:12:50,366
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing permission
for /tmp/hbase-hbase from 755 to 775
2017-04-25 14:12:50,368
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content':
StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2017-04-25 14:12:50,369
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
{'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
2017-04-25 14:12:50,393
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping
Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] due
to not_if
2017-04-25 14:12:50,394
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Group['hdfs'] {}
2017-04-25 14:12:50,395
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding group
Group['hdfs']
2017-04-25 14:12:50,430
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - User['hdfs']
{'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
2017-04-25 14:12:50,431
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying user hdfs
2017-04-25 14:12:50,474
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type:
2017-04-25 14:12:50,474
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Directory['/etc/hadoop'] {'mode': 0755}
2017-04-25 14:12:50,474
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating directory
Directory['/etc/hadoop'] since it doesn't exist.
2017-04-25 14:12:50,475
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner':
'hdfs', 'group': 'hadoop', 'mode': 01777}
2017-04-25 14:12:50,476
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating directory
Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] since it
doesn't exist.
2017-04-25 14:12:50,476
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing owner for
/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs
2017-04-25 14:12:50,476
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing group for
/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hadoop
2017-04-25 14:12:50,477
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing permission
for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 755 to 1777
2017-04-25 14:12:50,477
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/'] {'create_parents':
True}
2017-04-25 14:12:50,478
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating directory
Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/'] since it doesn't
exist.
2017-04-25 14:12:50,479
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> -
File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'] {'content':
DownloadSource('
http://css-fx-met-poc-82:8080/resources//jdk-8u77-linux-x64.tar.gz'),
'not_if': 'test -f /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'}
2017-04-25 14:12:50,504
<http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Downloading the
file from http://css-fx-met-poc-82:8080/resources//jdk-8u77-linux-x64.tar.gz
2017-04-25 14:12:53,793
<http://airmail.calendar/2017-04-25%2014:12:53%20EDT> -
Directory['/usr/jdk64'] {}
2017-04-25 14:12:53,793
<http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating directory
Directory['/usr/jdk64'] since it doesn't exist.
2017-04-25 14:12:53,794
<http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute[('chmod',
'a+x', u'/usr/jdk64')] {'sudo': True}
2017-04-25 14:12:53,824
<http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Execute['cd
/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf
/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz && ambari-sudo.sh cp
-rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {}
2017-04-25 14:13:00,439
<http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action': ['delete']}
2017-04-25 14:13:00,440
<http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing directory
Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and all its content
2017-04-25 14:13:00,620
<http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access': 'a'}
2017-04-25 14:13:00,621
<http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Execute[('chmod',
'-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True}
2017-04-25 14:13:00,682
<http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Initializing 2
repositories
2017-04-25 14:13:00,683
<http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
Repository['HDP-2.5'] {'base_url':
'http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0',
'action': ['create'], 'components': [u'HDP', 'main'], 'repo_template':
'[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
%}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
%}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP',
'mirror_list': None}
2017-04-25 14:13:00,709
<http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
2017-04-25 14:13:00,711
<http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist
2017-04-25 14:13:00,711
<http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
Repository['HDP-UTILS-1.1.0.21'] {'base_url':
'http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7',
'action': ['create'], 'components': [u'HDP-UTILS', 'main'],
'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
%}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
%}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS',
'mirror_list': None}
2017-04-25 14:13:00,717
<http://airmail.calendar/2017-04-25%2014:13:00%20EDT> -
File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content': InlineTemplate(...)}
2017-04-25 14:13:00,717
<http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing
File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist
2017-04-25 14:13:00,718
<http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Package['unzip']
{'retry_on_repo_unavailability': False, 'retry_count': 5}
2017-04-25 14:13:00,846
<http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing package
unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
2017-04-25 14:13:03,413
<http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Package['curl']
{'retry_on_repo_unavailability': False, 'retry_count': 5}
2017-04-25 14:13:03,429
<http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping
installation of existing package curl
2017-04-25 14:13:03,430
<http://airmail.calendar/2017-04-25%2014:13:03%20EDT> -
Package['hdp-select'] {'retry_on_repo_unavailability': False,
'retry_count': 5}
2017-04-25 14:13:03,451
<http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing package
hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select')
2017-04-25 14:13:04,948
<http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES Data Node
2017-04-25 14:13:04,952
<http://airmail.calendar/2017-04-25%2014:13:04%20EDT> -
Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability': False,
'retry_count': 5}
2017-04-25 14:13:05,086
<http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing package
elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install
elasticsearch-2.3.3')
Command failed after 1 tries
On April 25, 2017 at 13:47:17, Otto Fowler (ottobackwards@gmail.com) wrote:
Yes, I’m going to run through again and get the logs. The basic issue is,
no repo tab -> if I should have one,
and the ES yum command failing with nothing to do, which stops deployment
on all the nodes.
On April 25, 2017 at 13:36:25, Michael Miklavcic (
michael.miklavcic@gmail.com) wrote:
I've been doing an internal hwx deployment (HCP) recently for testing
purposes. I have some ES tweaks to work out, but other than that it has
been running smoothly. Can you share any log info or screenshots?
Mike
On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com> wrote:
> Hi,
> Should I see the repo tab in the ambari wizard for metron at this point?
> Asking for a friend who can’t get the wizard to work :)
>
Re: Ambari Wizard: Repo Tab
Posted by Otto Fowler <ot...@gmail.com>.
Yes, I’m going to run through again and get the logs. The basic issue is,
no repo tab -> if I should have one,
and the ES yum command failing with nothing to do, which stops deployment
on all the nodes.
On April 25, 2017 at 13:36:25, Michael Miklavcic (
michael.miklavcic@gmail.com) wrote:
I've been doing an internal hwx deployment (HCP) recently for testing
purposes. I have some ES tweaks to work out, but other than that it has
been running smoothly. Can you share any log info or screenshots?
Mike
On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com> wrote:
> Hi,
> Should I see the repo tab in the ambari wizard for metron at this point?
> Asking for a friend who can’t get the wizard to work :)
>
Re: Ambari Wizard: Repo Tab
Posted by Michael Miklavcic <mi...@gmail.com>.
I've been doing an internal hwx deployment (HCP) recently for testing
purposes. I have some ES tweaks to work out, but other than that it has
been running smoothly. Can you share any log info or screenshots?
Mike
On Apr 25, 2017 11:30 AM, "Otto Fowler" <ot...@gmail.com> wrote:
> Hi,
> Should I see the repo tab in the ambari wizard for metron at this point?
> Asking for a friend who can’t get the wizard to work :)
>