You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by Abid Zaidi <ab...@hotmail.com> on 2020/03/01 12:39:35 UTC
Failing to Setup Apache Metron Dev Machine using Vagrant (Require
Assistance and Guidance)
I am setting up centos7 single node VM through vagrant for Dev Environment,
I already tried centos6 from other branches and got no luck in that,
So I check out branch of METRON-2088-SUPPORT-HDP-3.1 and followed the ReadMe, before starting I ran platform.sh and everything looked good..
On Vagrant Up it just fails on Metron_config with message
[node1]: FAILED! => {"changed": false, "msg": "Request failed with status FAILED"}
At this point I could open ambari but all services were down and each of them said the same thing, Failed to install service, Below is the link to my logs:
https://community.cloudera.com/t5/Support-Questions/Failing-to-Setup-Apache-Metron-Dev-Machine-using-Vagrant/m-p/290740/highlight/true#M215064
This is the furthest I've come to deploying, even with the CentOS 6 I came this far,
I then also tried provision with following command,
env ANSIBLE_ARGS=' -vvvv' vagrant --ansible-tags="hdp-deploy" provision
I've done this 2-3 times and I reach to this same point,
I need to setup a Dev Environment where I could add a few more features in Apache Metron's REST API with UI pages on the front-end. I seem to be far away from my targets as I am unable to start up the vagrant machine, It would be good if a Metron Developer could guide me with what I'm trying to achieve.
Regards,
Abid Zaidi.
abidzaidi7@hotmail.com
Re: Failing to Setup Apache Metron Dev Machine using Vagrant (Require
Assistance and Guidance)
Posted by Otto Fowler <ot...@gmail.com>.
I have tried this on master, and progressed all the way to the failure to
setup enrichment because of the geomind change.
I believe that this issue is only with the 3.1 feature branch.
On March 3, 2020 at 09:49:39, Michael Miklavcic (michael.miklavcic@gmail.com)
wrote:
It looks like the binaries are no longer available? I've not encountered
this before. Not sure if this has anything to do with it (from Cloudera's
publications -
https://www.cloudera.com/content/dam/www/marketing/documents/campaign/451-cloudera-opens-up-about-opensource.pdf
)
On Mon, Mar 2, 2020 at 10:28 PM Abid Zaidi <ab...@hotmail.com> wrote:
> Thanks Michael,
> I am using Ubuntu 19.10, 32GB RAM, i7 (12 cores),
> Vagrant machine is obviously on CentOS 7, I've given it 16GB RAM.
>
> On investigation the issue was with a Yum repo , the error is following
>
> ....
> RuntimeError: Failed to execute command '/usr/bin/yum -y install unzip',
> exited with code '1', message: '
>
https://archive.cloudera.com/p/HDP/centos7/3.x/updates/3.1.5.0/repodata/repomd.xml:
> [Errno 14] HTTPS Error 401 - Unauthorized
>
> Trying other mirror.
>
> One of the configured repositories failed (*HDP-3.1-repo-1*),
> ....
>
>
> I have searched the available repos but apparently for HDP 3.1.5 (
> reference
> <
https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/bk_ambari-installation/content/ch03s02s01.html>)
> it requires credentials. (e.g *
https://username:password@archive.cloudera.com/p/HDP/centos7/3.x/updates/3.1.5.0
> <
https://username:password@archive.cloudera.com/p/HDP/centos7/3.x/updates/3.1.5.0>)*
>
> For acquiring these I'd have to file a case on cloudera support, but when
> i reach there it says: *'Access to page is restricted'. *Later I got to
> know that I have to purchase their product since it is behind a payment
> wall (reference
> <
https://community.cloudera.com/t5/Support-Questions/Unable-to-access-Ambari-repo/td-p/286813>
> ).
>
> If you have any information of a work around do let me know..
>
> ------------------------------
> *From:* Michael Miklavcic <mi...@gmail.com>
> *Sent:* Monday, March 2, 2020 10:07 AM
> *To:* dev@metron.apache.org <de...@metron.apache.org>
> *Subject:* Re: Failing to Setup Apache Metron Dev Machine using Vagrant
> (Require Assistance and Guidance)
>
> Head over to Ambari http://node1:8080 and click the activity icon (should
> be somewhere along the top menu bar). You should be able to see more
> details about what Ambari was doing at the time of failure. It's possible
> there were just some minor issues in starting up the services. If that's
> the case, then I think all you'll need to do is start them up manually.
>
> FYI, HDP 3.1 should be deployed on CentOS 7 only. Also, the Vagrant
> environment should have at least 8 GB of mem allocated for it (that's our
> default).
>
> On Sun, Mar 1, 2020 at 5:39 AM Abid Zaidi <ab...@hotmail.com> wrote:
>
> > I am setting up centos7 single node VM through vagrant for Dev
> Environment,
> >
> >
> > I already tried centos6 from other branches and got no luck in that,
> >
> > So I check out branch of METRON-2088-SUPPORT-HDP-3.1 and followed the
> > ReadMe, before starting I ran platform.sh and everything looked good..
> >
> > On Vagrant Up it just fails on Metron_config with message
> >
> > [node1]: FAILED! => {"changed": false, "msg": "Request failed with
status
> > FAILED"}
> >
> > At this point I could open ambari but all services were down and each
of
> > them said the same thing, Failed to install service, Below is the link
to
> > my logs:
> >
> >
> >
>
https://community.cloudera.com/t5/Support-Questions/Failing-to-Setup-Apache-Metron-Dev-Machine-using-Vagrant/m-p/290740/highlight/true#M215064
> >
> >
> > This is the furthest I've come to deploying, even with the CentOS 6 I
> came
> > this far,
> >
> > I then also tried provision with following command,
> >
> > env ANSIBLE_ARGS=' -vvvv' vagrant --ansible-tags="hdp-deploy" provision
> >
> >
> > I've done this 2-3 times and I reach to this same point,
> >
> > I need to setup a Dev Environment where I could add a few more features
> in
> > Apache Metron's REST API with UI pages on the front-end. I seem to be
far
> > away from my targets as I am unable to start up the vagrant machine, It
> > would be good if a Metron Developer could guide me with what I'm trying
> to
> > achieve.
> >
> >
> > Regards,
> >
> > Abid Zaidi.
> >
> > abidzaidi7@hotmail.com
> >
> >
> >
>
Re: Failing to Setup Apache Metron Dev Machine using Vagrant (Require
Assistance and Guidance)
Posted by Michael Miklavcic <mi...@gmail.com>.
It looks like the binaries are no longer available? I've not encountered
this before. Not sure if this has anything to do with it (from Cloudera's
publications -
https://www.cloudera.com/content/dam/www/marketing/documents/campaign/451-cloudera-opens-up-about-opensource.pdf
)
On Mon, Mar 2, 2020 at 10:28 PM Abid Zaidi <ab...@hotmail.com> wrote:
> Thanks Michael,
> I am using Ubuntu 19.10, 32GB RAM, i7 (12 cores),
> Vagrant machine is obviously on CentOS 7, I've given it 16GB RAM.
>
> On investigation the issue was with a Yum repo , the error is following
>
> ....
> RuntimeError: Failed to execute command '/usr/bin/yum -y install unzip',
> exited with code '1', message: '
> https://archive.cloudera.com/p/HDP/centos7/3.x/updates/3.1.5.0/repodata/repomd.xml:
> [Errno 14] HTTPS Error 401 - Unauthorized
>
> Trying other mirror.
>
> One of the configured repositories failed (*HDP-3.1-repo-1*),
> ....
>
>
> I have searched the available repos but apparently for HDP 3.1.5 (
> reference
> <https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/bk_ambari-installation/content/ch03s02s01.html>)
> it requires credentials. (e.g *https://username:password@archive.cloudera.com/p/HDP/centos7/3.x/updates/3.1.5.0
> <https://username:password@archive.cloudera.com/p/HDP/centos7/3.x/updates/3.1.5.0>)*
>
> For acquiring these I'd have to file a case on cloudera support, but when
> i reach there it says: *'Access to page is restricted'. *Later I got to
> know that I have to purchase their product since it is behind a payment
> wall (reference
> <https://community.cloudera.com/t5/Support-Questions/Unable-to-access-Ambari-repo/td-p/286813>
> ).
>
> If you have any information of a work around do let me know..
>
> ------------------------------
> *From:* Michael Miklavcic <mi...@gmail.com>
> *Sent:* Monday, March 2, 2020 10:07 AM
> *To:* dev@metron.apache.org <de...@metron.apache.org>
> *Subject:* Re: Failing to Setup Apache Metron Dev Machine using Vagrant
> (Require Assistance and Guidance)
>
> Head over to Ambari http://node1:8080 and click the activity icon (should
> be somewhere along the top menu bar). You should be able to see more
> details about what Ambari was doing at the time of failure. It's possible
> there were just some minor issues in starting up the services. If that's
> the case, then I think all you'll need to do is start them up manually.
>
> FYI, HDP 3.1 should be deployed on CentOS 7 only. Also, the Vagrant
> environment should have at least 8 GB of mem allocated for it (that's our
> default).
>
> On Sun, Mar 1, 2020 at 5:39 AM Abid Zaidi <ab...@hotmail.com> wrote:
>
> > I am setting up centos7 single node VM through vagrant for Dev
> Environment,
> >
> >
> > I already tried centos6 from other branches and got no luck in that,
> >
> > So I check out branch of METRON-2088-SUPPORT-HDP-3.1 and followed the
> > ReadMe, before starting I ran platform.sh and everything looked good..
> >
> > On Vagrant Up it just fails on Metron_config with message
> >
> > [node1]: FAILED! => {"changed": false, "msg": "Request failed with status
> > FAILED"}
> >
> > At this point I could open ambari but all services were down and each of
> > them said the same thing, Failed to install service, Below is the link to
> > my logs:
> >
> >
> >
> https://community.cloudera.com/t5/Support-Questions/Failing-to-Setup-Apache-Metron-Dev-Machine-using-Vagrant/m-p/290740/highlight/true#M215064
> >
> >
> > This is the furthest I've come to deploying, even with the CentOS 6 I
> came
> > this far,
> >
> > I then also tried provision with following command,
> >
> > env ANSIBLE_ARGS=' -vvvv' vagrant --ansible-tags="hdp-deploy" provision
> >
> >
> > I've done this 2-3 times and I reach to this same point,
> >
> > I need to setup a Dev Environment where I could add a few more features
> in
> > Apache Metron's REST API with UI pages on the front-end. I seem to be far
> > away from my targets as I am unable to start up the vagrant machine, It
> > would be good if a Metron Developer could guide me with what I'm trying
> to
> > achieve.
> >
> >
> > Regards,
> >
> > Abid Zaidi.
> >
> > abidzaidi7@hotmail.com
> >
> >
> >
>
Re: Failing to Setup Apache Metron Dev Machine using Vagrant (Require
Assistance and Guidance)
Posted by Abid Zaidi <ab...@hotmail.com>.
Thanks Michael,
I am using Ubuntu 19.10, 32GB RAM, i7 (12 cores),
Vagrant machine is obviously on CentOS 7, I've given it 16GB RAM.
On investigation the issue was with a Yum repo , the error is following
....
RuntimeError: Failed to execute command '/usr/bin/yum -y install unzip', exited with code '1', message: 'https://archive.cloudera.com/p/HDP/centos7/3.x/updates/3.1.5.0/repodata/repomd.xml: [Errno 14] HTTPS Error 401 - Unauthorized
Trying other mirror.
One of the configured repositories failed (HDP-3.1-repo-1),
....
I have searched the available repos but apparently for HDP 3.1.5 (reference<https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/bk_ambari-installation/content/ch03s02s01.html>) it requires credentials. (e.g https://username:password@archive.cloudera.com/p/HDP/centos7/3.x/updates/3.1.5.0)
For acquiring these I'd have to file a case on cloudera support, but when i reach there it says: 'Access to page is restricted'. Later I got to know that I have to purchase their product since it is behind a payment wall (reference<https://community.cloudera.com/t5/Support-Questions/Unable-to-access-Ambari-repo/td-p/286813>).
If you have any information of a work around do let me know..
________________________________
From: Michael Miklavcic <mi...@gmail.com>
Sent: Monday, March 2, 2020 10:07 AM
To: dev@metron.apache.org <de...@metron.apache.org>
Subject: Re: Failing to Setup Apache Metron Dev Machine using Vagrant (Require Assistance and Guidance)
Head over to Ambari http://node1:8080 and click the activity icon (should
be somewhere along the top menu bar). You should be able to see more
details about what Ambari was doing at the time of failure. It's possible
there were just some minor issues in starting up the services. If that's
the case, then I think all you'll need to do is start them up manually.
FYI, HDP 3.1 should be deployed on CentOS 7 only. Also, the Vagrant
environment should have at least 8 GB of mem allocated for it (that's our
default).
On Sun, Mar 1, 2020 at 5:39 AM Abid Zaidi <ab...@hotmail.com> wrote:
> I am setting up centos7 single node VM through vagrant for Dev Environment,
>
>
> I already tried centos6 from other branches and got no luck in that,
>
> So I check out branch of METRON-2088-SUPPORT-HDP-3.1 and followed the
> ReadMe, before starting I ran platform.sh and everything looked good..
>
> On Vagrant Up it just fails on Metron_config with message
>
> [node1]: FAILED! => {"changed": false, "msg": "Request failed with status
> FAILED"}
>
> At this point I could open ambari but all services were down and each of
> them said the same thing, Failed to install service, Below is the link to
> my logs:
>
>
> https://community.cloudera.com/t5/Support-Questions/Failing-to-Setup-Apache-Metron-Dev-Machine-using-Vagrant/m-p/290740/highlight/true#M215064
>
>
> This is the furthest I've come to deploying, even with the CentOS 6 I came
> this far,
>
> I then also tried provision with following command,
>
> env ANSIBLE_ARGS=' -vvvv' vagrant --ansible-tags="hdp-deploy" provision
>
>
> I've done this 2-3 times and I reach to this same point,
>
> I need to setup a Dev Environment where I could add a few more features in
> Apache Metron's REST API with UI pages on the front-end. I seem to be far
> away from my targets as I am unable to start up the vagrant machine, It
> would be good if a Metron Developer could guide me with what I'm trying to
> achieve.
>
>
> Regards,
>
> Abid Zaidi.
>
> abidzaidi7@hotmail.com
>
>
>
Re: Failing to Setup Apache Metron Dev Machine using Vagrant (Require
Assistance and Guidance)
Posted by Michael Miklavcic <mi...@gmail.com>.
Head over to Ambari http://node1:8080 and click the activity icon (should
be somewhere along the top menu bar). You should be able to see more
details about what Ambari was doing at the time of failure. It's possible
there were just some minor issues in starting up the services. If that's
the case, then I think all you'll need to do is start them up manually.
FYI, HDP 3.1 should be deployed on CentOS 7 only. Also, the Vagrant
environment should have at least 8 GB of mem allocated for it (that's our
default).
On Sun, Mar 1, 2020 at 5:39 AM Abid Zaidi <ab...@hotmail.com> wrote:
> I am setting up centos7 single node VM through vagrant for Dev Environment,
>
>
> I already tried centos6 from other branches and got no luck in that,
>
> So I check out branch of METRON-2088-SUPPORT-HDP-3.1 and followed the
> ReadMe, before starting I ran platform.sh and everything looked good..
>
> On Vagrant Up it just fails on Metron_config with message
>
> [node1]: FAILED! => {"changed": false, "msg": "Request failed with status
> FAILED"}
>
> At this point I could open ambari but all services were down and each of
> them said the same thing, Failed to install service, Below is the link to
> my logs:
>
>
> https://community.cloudera.com/t5/Support-Questions/Failing-to-Setup-Apache-Metron-Dev-Machine-using-Vagrant/m-p/290740/highlight/true#M215064
>
>
> This is the furthest I've come to deploying, even with the CentOS 6 I came
> this far,
>
> I then also tried provision with following command,
>
> env ANSIBLE_ARGS=' -vvvv' vagrant --ansible-tags="hdp-deploy" provision
>
>
> I've done this 2-3 times and I reach to this same point,
>
> I need to setup a Dev Environment where I could add a few more features in
> Apache Metron's REST API with UI pages on the front-end. I seem to be far
> away from my targets as I am unable to start up the vagrant machine, It
> would be good if a Metron Developer could guide me with what I'm trying to
> achieve.
>
>
> Regards,
>
> Abid Zaidi.
>
> abidzaidi7@hotmail.com
>
>
>