You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@metron.apache.org by Scott Cote <sc...@lucidworks.com> on 2018/10/24 22:03:14 UTC
Building Metron with Vagrant from Master Branch on Mac OS X using
SOLR
Hello Fellow Metronites:
As I explored my very first setup of Metron for the vagrant install, I ran into a few hiccups.
I have listed below a summary of a rather long email with the details broken out. Apologies for the lengthiness.
1. Hiccups while performing the pip install ansible step
2. install Xcode command line tools
3. install Xcode command line tools
4. After steps 1 and 2 were performed, I did a build from master and attempted to startup the vagrant instance. Below is a dump of that attempts. Vagrant still failed to come up. More fixes:
5. enable solr and disable elastic
6. make install verbose
7. ansible correction based on feedback that I'm using head of master and its not in synch with the specified version of ansible on the page
1 Hiccups while performing the pip install ansible step:
```I was able to get this to work by upgrading openssl using brew
brew upgrade openssl
and then setting the CPPFLAGS and LDFLAGS to point to the openssl lib from brew
export CPPFLAGS=-I/usr/local/opt/openssl/include
export LDFLAGS=-L/usr/local/opt/openssl/lib```
when installing Vagrant at the step `pip install ansible==2.2.2.0`
i encountered problems with the crypt... python packages.
had to ensure that `brew upgrade openssl`
was performed.
2. install Xcode command line tools (again for some reason???)
`$ xcode-select --install`
3. errors that occurred while sensible installed with bad SSL brew casks. - (see step 1):
```creating build/temp.macosx-10.13-intel-2.7/build/temp.macosx-10.13-intel-2.7
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c build/temp.macosx-10.13-intel-2.7/_openssl.c -o build/temp.macosx-10.13-intel-2.7/build/temp.macosx-10.13-intel-2.7/_openssl.o -Wconversion -Wno-error=sign-conversion
build/temp.macosx-10.13-intel-2.7/_openssl.c:498:10: fatal error: 'openssl/opensslv.h' file not found
#include <openssl/opensslv.h>
^~~~~~~~~~~~~~~~~~~~
1 error generated.
error: command 'cc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/vx/n3g3g7k94lg8pxz1hdp4bhb40000gp/T/pip-build-uWUxRG/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/vx/n3g3g7k94lg8pxz1hdp4bhb40000gp/T/pip-OpX1S3-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/vx/n3g3g7k94lg8pxz1hdp4bhb40000gp/T/pip-build-uWUxRG/cryptography```
4. After steps 1 and 2 were performed, I did a build from master and attempted to startup the vagrant instance. Below is a dump of that attempts. Vagrant still failed to come up. More fixes:
Tried to just use vagrant up and got a fail ....
This snippet has the following pieces of information to help you help me:
4.1. information about my machine/env
4.2. overview of sequence of commands to reproduce
4.3. terminal dump
Here are the details:
4.1. Information about my machine
4.1.1. unamme output:
~/workspace/metron/metron-deployment/development/centos6 (master) $ uname -a
Darwin ScottCotesLWMBP.fios-router.home 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64
4.1.2. platform-info.sh output
~/workspace/metron/metron-deployment/scripts (master) $ ./platform-info.sh
Metron 0.6.1
--
* master
--
commit 0ba18026222f004316b96633780509c71934121e (HEAD -> master, origin/master, origin/HEAD)
Author: JonZeolla <ze...@gmail.com>
Date: Mon Oct 22 19:48:56 2018 -0400
METRON-1826 Update librdkafka and devtoolset (JonZeolla via jonzeolla) closes apache/metron#1238
--
metron-deployment/development/centos6/Vagrantfile | 1 +
metron-deployment/development/centos6/ansible/inventory/group_vars/all | 4 ++--
2 files changed, 3 insertions(+), 2 deletions(-)
--
ansible 2.2.2.0
config file =
configured module search path = Default w/o overrides
--
Vagrant 2.1.5
--
vagrant-hostmanager (1.8.9, global)
--
Python 2.7.10
--
Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-03T14:39:06-05:00)
Maven home: /usr/local/Cellar/maven/3.5.0/libexec
Java version: 1.8.0_181, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac"
--
Docker version 18.06.1-ce, build e68fc7a
--
node
v10.7.0
--
npm
6.2.0
--
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 10.0.0 (clang-1000.11.45.2)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
--
Compiler is C++11 compliant
--
Darwin ScottCotesLWMBP.fios-router.home 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64
--
Total System Memory = 16384 MB
Processor Model: Intel(R) Core(TM) i7-4770HQ CPU
Processor Speed: 2.20GHz
Total Physical Processors: 4
Total cores: 4
Disk information:
/dev/disk1s1 466Gi 408Gi 50Gi 90% 3427183 9223372036851348624 0% /
/dev/disk1s4 466Gi 7.0Gi 50Gi 13% 7 9223372036854775800 0% /private/var/vm
This CPU appears to support virtualization
4.2. overview of sequence of commands to reproduce
followed instructions for vagrant setup with the last following steps:
4.2.1. ~/workspace/metron/metron-deployment/development/centos6 (master) $ vagrant destroy -f
4.2.2. ~/workspace/metron/metron-deployment/development/centos6 (master) $ cd ../../..
4.2.3. ~/workspace/metron (master) $ git pull
4.2.4. ~/workspace/metron (master) $ mvn clean install -DskipTests -T C4
4.2.5. ~/workspace/metron (master) $ cd metron-deployment/development/centos6
4.2.4. ~/workspace/metron/metron-deployment/development/centos6 (master) $ vagrant up
4.3. terminal dump
...................................................
..--->>>redacted terminal dump for brevity<<<--- ..
...................................................
[INFO] --- maven-install-plugin:2.4:install (default-install) @ metron-rest ---
[INFO] Installing /Users/scottcote/workspace/metron/metron-interface/metron-rest/target/metron-rest-0.6.1.jar to /Users/scottcote/.m2/repository/org/apache/metron/metron-rest/0.6.1/metron-rest-0.6.1.jar
[INFO] Installing /Users/scottcote/workspace/metron/metron-interface/metron-rest/pom.xml to /Users/scottcote/.m2/repository/org/apache/metron/metron-rest/0.6.1/metron-rest-0.6.1.pom
[INFO] Installing /Users/scottcote/workspace/metron/metron-interface/metron-rest/target/metron-rest-0.6.1-archive.tar.gz to /Users/scottcote/.m2/repository/org/apache/metron/metron-rest/0.6.1/metron-rest-0.6.1-archive.tar.gz
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Metron ............................................. SUCCESS [ 14.439 s]
[INFO] metron-stellar ..................................... SUCCESS [ 1.317 s]
[INFO] stellar-common ..................................... SUCCESS [ 39.920 s]
[INFO] metron-analytics ................................... SUCCESS [ 1.221 s]
[INFO] metron-maas-common ................................. SUCCESS [ 1.142 s]
[INFO] metron-platform .................................... SUCCESS [ 1.666 s]
[INFO] metron-zookeeper ................................... SUCCESS [ 5.004 s]
[INFO] metron-test-utilities .............................. SUCCESS [ 2.454 s]
[INFO] metron-integration-test ............................ SUCCESS [ 20.909 s]
[INFO] metron-maas-service ................................ SUCCESS [ 20.476 s]
[INFO] metron-common ...................................... SUCCESS [ 34.899 s]
[INFO] metron-statistics .................................. SUCCESS [ 33.713 s]
[INFO] metron-writer ...................................... SUCCESS [ 32.067 s]
[INFO] metron-storm-kafka-override ........................ SUCCESS [ 11.703 s]
[INFO] metron-storm-kafka ................................. SUCCESS [ 2.069 s]
[INFO] metron-hbase ....................................... SUCCESS [ 3.316 s]
[INFO] metron-profiler-common ............................. SUCCESS [ 2.298 s]
[INFO] metron-profiler-client ............................. SUCCESS [ 31.861 s]
[INFO] metron-profiler-storm .............................. SUCCESS [01:24 min]
[INFO] metron-profiler-spark .............................. SUCCESS [01:26 min]
[INFO] metron-profiler-repl ............................... SUCCESS [ 22.633 s]
[INFO] metron-hbase-client ................................ SUCCESS [ 23.168 s]
[INFO] metron-enrichment .................................. SUCCESS [01:27 min]
[INFO] metron-indexing .................................... SUCCESS [ 23.352 s]
[INFO] metron-solr ........................................ SUCCESS [01:06 min]
[INFO] metron-job ......................................... SUCCESS [ 1.818 s]
[INFO] metron-pcap ........................................ SUCCESS [ 4.094 s]
[INFO] metron-parsers ..................................... SUCCESS [01:14 min]
[INFO] metron-pcap-backend ................................ SUCCESS [ 54.775 s]
[INFO] metron-data-management ............................. SUCCESS [01:13 min]
[INFO] metron-management .................................. SUCCESS [ 12.919 s]
[INFO] elasticsearch-shaded ............................... SUCCESS [ 25.123 s]
[INFO] metron-elasticsearch ............................... SUCCESS [01:22 min]
[INFO] metron-deployment .................................. SUCCESS [ 1.265 s]
[INFO] Elasticsearch Ambari Management Pack ............... SUCCESS [ 2.529 s]
[INFO] Metron Ambari Management Pack ...................... SUCCESS [ 3.869 s]
[INFO] metron-contrib ..................................... SUCCESS [ 1.151 s]
[INFO] metron-docker ...................................... SUCCESS [ 4.692 s]
[INFO] metron-performance ................................. SUCCESS [ 4.457 s]
[INFO] metron-interface ................................... SUCCESS [ 9.772 s]
[INFO] metron-config ...................................... SUCCESS [05:00 min]
[INFO] metron-alerts ...................................... SUCCESS [03:48 min]
[INFO] metron-rest-client ................................. SUCCESS [ 1.148 s]
[INFO] metron-rest ........................................ SUCCESS [ 33.988 s]
[INFO] site-book .......................................... SUCCESS [ 1.929 s]
[INFO] 3rd party Functions (just for tests) ............... SUCCESS [ 0.583 s]
[INFO] stellar-zeppelin ................................... SUCCESS [ 2.079 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:46 min (Wall Clock)
[INFO] Finished at: 2018-10-23T09:22:06-05:00
[INFO] Final Memory: 220M/2041M
[INFO] ------------------------------------------------------------------------
~/workspace/metron (master) $ cd metron-deployment/development/centos6/
~/workspace/metron/metron-deployment/development/centos6 (master) $ vagrant up
Running with ansible-skip-tags: ["sensors"]
Bringing machine 'node1' up with 'virtualbox' provider...
==> node1: Importing base box 'centos/6'...
==> node1: Matching MAC address for NAT networking...
==> node1: Checking if box 'centos/6' is up to date...
==> node1: Setting the name of the VM: centos6_node1_1540304760857_3427
==> node1: Clearing any previously set network interfaces...
==> node1: Preparing network interfaces based on configuration...
node1: Adapter 1: nat
node1: Adapter 2: hostonly
==> node1: Forwarding ports...
node1: 22 (guest) => 2222 (host) (adapter 1)
==> node1: Running 'pre-boot' VM customizations...
==> node1: Booting VM...
==> node1: Waiting for machine to boot. This may take a few minutes...
node1: SSH address: 127.0.0.1:2222
node1: SSH username: vagrant
node1: SSH auth method: private key
node1:
node1: Vagrant insecure key detected. Vagrant will automatically replace
node1: this with a newly generated keypair for better security.
node1:
node1: Inserting generated public key within guest...
node1: Removing insecure key from the guest if it's present...
node1: Key inserted! Disconnecting and reconnecting using new SSH key...
==> node1: Machine booted and ready!
==> node1: Checking for guest additions in VM...
node1: No guest additions were detected on the base box for this VM! Guest
node1: additions are required for forwarded ports, shared folders, host only
node1: networking, and more. If SSH fails on this machine, please install
node1: the guest additions and repackage the box to continue.
node1:
node1: This is not an error message; everything may continue to work properly,
node1: in which case you may ignore this message.
==> node1: Setting hostname...
==> node1: Configuring and enabling network interfaces...
node1: SSH address: 127.0.0.1:2222
node1: SSH username: vagrant
node1: SSH auth method: private key
==> node1: Rsyncing folder: /Users/scottcote/workspace/metron/metron-deployment/development/centos6/ => /vagrant
==> node1: [vagrant-hostmanager:guests] Updating hosts file on active guest virtual machines...
==> node1: [vagrant-hostmanager:host] Updating hosts file on your workstation (password may be required)...
Password:
==> node1: Running provisioner: ansible...
Vagrant has automatically selected the compatibility mode '2.0'
according to the Ansible version installed (2.2.2.0).
Alternatively, the compatibility mode can be specified in your Vagrantfile:
https://www.vagrantup.com/docs/provisioning/ansible_common.html#compatibility_mode
node1: Running ansible-playbook...
PYTHONUNBUFFERED=1 ANSIBLE_FORCE_COLOR=true ANSIBLE_HOST_KEY_CHECKING=false ANSIBLE_SSH_ARGS='-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o IdentityFile=/Users/scottcote/workspace/metron/metron-deployment/development/centos6/.vagrant/machines/node1/virtualbox/private_key -o ControlMaster=auto -o ControlPersist=60s' ansible-playbook --connection=ssh --timeout=30 --extra-vars=ansible_user\=\'vagrant\' --limit="node1" --inventory-file=ansible/inventory --become -vvv --skip-tags=sensors ansible/playbook.yml
Using /Users/scottcote/workspace/metron/metron-deployment/development/centos6/ansible.cfg as config file
ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.
The error appears to have been in '/Users/scottcote/workspace/metron/metron-deployment/ansible/roles/enable-swap/tasks/main.yml': line 29, column 3, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- include_tasks: enable-swap.yml
^ here
The error appears to have been in '/Users/scottcote/workspace/metron/metron-deployment/ansible/roles/enable-swap/tasks/main.yml': line 29, column 3, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- include_tasks: enable-swap.yml
^ here
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.
5. enable solr and disable elastic
go to `ansible/inventory/group_vars/all`
and set
`# search
install_elasticsearch: False
install_solr: True
solr_collection_name: Metron
solr_number_shards: 1
solr_replication_factor: 1
elasticsearch_transport_port: 9300
## The elasticsearch_network_interface must be in the form specified for Elasticsearch, with leading and trailing underscores.
elasticsearch_network_interface: _eth1_
elasticsearch_web_port: 9200
ansible/inventory/group_vars/all
`
6 make install verbose
mod this file ~/devprojects/metron/metron-deployment/development/centos6/Vagrantfile
er, metron/metron-deployment/development/centos6/Vagrantfile
under config.vm.provision add this
ansible.verbose = "vvv"
7. ansible correction based on feedback that I'm using head of master and its not in synch with the specified version of ansible on the page
`pip install ansible==2.5.0`
then do `vagrant up`
Re: Building Metron with Vagrant from Master Branch on Mac OS X using
SOLR
Posted by Scott Cote <sc...@lucidworks.com>.
Thank you for your help now and earlier with Ward :)
The talk at Activate went off very smooth - am looking forward to digging in.
Your summary => I think that was the gist of it.
Not sure if its a mac problem or not, but don't leave out the brew upgrade of openssl and the setting of those environment variables. Maybe they were only a problem for ansible 2.2.2.0
Let me know if you want me to review document changes - it would help me better understand ya'lls documentation style. Kinda an easy way to start learning how to be a contributor. We had a whole talk at the activate conference on how to be a committer and documentation was one of the talking points.
SCott
> On Oct 24, 2018, at 5:48 PM, Michael Miklavcic <mi...@gmail.com> wrote:
>
> Thanks for the feedback Scott. I see 3 categories:
> 1. Some setup and prereq doc clarification for newcomers, e.g. need for xcode install, clarify doc version compatibility
> 2. Added details for configuring Solr instead of ES
> 3. Some added details around self-help on debugging the install, e.g. enabling vvv with ansible (I was actually the dev that worked through that with Ward), some suggestion about finding errors in Ambari.
>
> Some of this could probably be simple Ansible enhancements. Have I missed anything? Are you still in the middle of getting full dev up or is it working for you now?
>
> Best,
> Mike Miklavcic
>
> On Oct 24, 2018 4:03 PM, "Scott Cote" <scott.cote@lucidworks.com <ma...@lucidworks.com>> wrote:
> Hello Fellow Metronites:
>
> As I explored my very first setup of Metron for the vagrant install, I ran into a few hiccups.
>
> I have listed below a summary of a rather long email with the details broken out. Apologies for the lengthiness.
>
> 1. Hiccups while performing the pip install ansible step
> 2. install Xcode command line tools
> 3. install Xcode command line tools
> 4. After steps 1 and 2 were performed, I did a build from master and attempted to startup the vagrant instance. Below is a dump of that attempts. Vagrant still failed to come up. More fixes:
> 5. enable solr and disable elastic
> 6. make install verbose
> 7. ansible correction based on feedback that I'm using head of master and its not in synch with the specified version of ansible on the page
>
> 1 Hiccups while performing the pip install ansible step:
>
> ```I was able to get this to work by upgrading openssl using brew
>
> brew upgrade openssl
>
> and then setting the CPPFLAGS and LDFLAGS to point to the openssl lib from brew
>
> export CPPFLAGS=-I/usr/local/opt/openssl/include
> export LDFLAGS=-L/usr/local/opt/openssl/lib```
>
> when installing Vagrant at the step `pip install ansible==2.2.2.0`
> i encountered problems with the crypt... python packages.
> had to ensure that `brew upgrade openssl`
> was performed.
>
>
> 2. install Xcode command line tools (again for some reason???)
>
> `$ xcode-select --install`
>
> 3. errors that occurred while sensible installed with bad SSL brew casks. - (see step 1):
> ```creating build/temp.macosx-10.13-intel-2.7/build/temp.macosx-10.13-intel-2.7
> cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c build/temp.macosx-10.13-intel-2.7/_openssl.c -o build/temp.macosx-10.13-intel-2.7/build/temp.macosx-10.13-intel-2.7/_openssl.o -Wconversion -Wno-error=sign-conversion
> build/temp.macosx-10.13-intel-2.7/_openssl.c:498:10: fatal error: 'openssl/opensslv.h' file not found
> #include <openssl/opensslv.h>
> ^~~~~~~~~~~~~~~~~~~~
> 1 error generated.
> error: command 'cc' failed with exit status 1
>
> ----------------------------------------
> Command "/usr/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/vx/n3g3g7k94lg8pxz1hdp4bhb40000gp/T/pip-build-uWUxRG/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/vx/n3g3g7k94lg8pxz1hdp4bhb40000gp/T/pip-OpX1S3-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/vx/n3g3g7k94lg8pxz1hdp4bhb40000gp/T/pip-build-uWUxRG/cryptography```
>
>
> 4. After steps 1 and 2 were performed, I did a build from master and attempted to startup the vagrant instance. Below is a dump of that attempts. Vagrant still failed to come up. More fixes:
>
> Tried to just use vagrant up and got a fail ....
> This snippet has the following pieces of information to help you help me:
> 4.1. information about my machine/env
> 4.2. overview of sequence of commands to reproduce
> 4.3. terminal dump
> Here are the details:
> 4.1. Information about my machine
> 4.1.1. unamme output:
> ~/workspace/metron/metron-deployment/development/centos6 (master) $ uname -a
> Darwin ScottCotesLWMBP.fios-router.home 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64
> 4.1.2. platform-info.sh output
> ~/workspace/metron/metron-deployment/scripts (master) $ ./platform-info.sh
> Metron 0.6.1
> --
> * master
> --
> commit 0ba18026222f004316b96633780509c71934121e (HEAD -> master, origin/master, origin/HEAD)
> Author: JonZeolla <zeolla@gmail.com <ma...@gmail.com>>
> Date: Mon Oct 22 19:48:56 2018 -0400
> METRON-1826 Update librdkafka and devtoolset (JonZeolla via jonzeolla) closes apache/metron#1238
> --
> metron-deployment/development/centos6/Vagrantfile | 1 +
> metron-deployment/development/centos6/ansible/inventory/group_vars/all | 4 ++--
> 2 files changed, 3 insertions(+), 2 deletions(-)
> --
> ansible 2.2.2.0
> config file =
> configured module search path = Default w/o overrides
> --
> Vagrant 2.1.5
> --
> vagrant-hostmanager (1.8.9, global)
> --
> Python 2.7.10
> --
> Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-03T14:39:06-05:00)
> Maven home: /usr/local/Cellar/maven/3.5.0/libexec
> Java version: 1.8.0_181, vendor: Oracle Corporation
> Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac"
> --
> Docker version 18.06.1-ce, build e68fc7a
> --
> node
> v10.7.0
> --
> npm
> 6.2.0
> --
> Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
> Apple LLVM version 10.0.0 (clang-1000.11.45.2)
> Target: x86_64-apple-darwin17.7.0
> Thread model: posix
> InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
> --
> Compiler is C++11 compliant
> --
> Darwin ScottCotesLWMBP.fios-router.home 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64
> --
> Total System Memory = 16384 MB
> Processor Model: Intel(R) Core(TM) i7-4770HQ CPU
> Processor Speed: 2.20GHz
> Total Physical Processors: 4
> Total cores: 4
> Disk information:
> /dev/disk1s1 466Gi 408Gi 50Gi 90% 3427183 9223372036851348624 0% /
> /dev/disk1s4 466Gi 7.0Gi 50Gi 13% 7 9223372036854775800 0% /private/var/vm
> This CPU appears to support virtualization
>
>
> 4.2. overview of sequence of commands to reproduce
> followed instructions for vagrant setup with the last following steps:
> 4.2.1. ~/workspace/metron/metron-deployment/development/centos6 (master) $ vagrant destroy -f
> 4.2.2. ~/workspace/metron/metron-deployment/development/centos6 (master) $ cd ../../..
> 4.2.3. ~/workspace/metron (master) $ git pull
> 4.2.4. ~/workspace/metron (master) $ mvn clean install -DskipTests -T C4
> 4.2.5. ~/workspace/metron (master) $ cd metron-deployment/development/centos6
> 4.2.4. ~/workspace/metron/metron-deployment/development/centos6 (master) $ vagrant up
>
> 4.3. terminal dump
> ...................................................
> ..--->>>redacted terminal dump for brevity<<<--- ..
> ...................................................
> [INFO] --- maven-install-plugin:2.4:install (default-install) @ metron-rest ---
> [INFO] Installing /Users/scottcote/workspace/metron/metron-interface/metron-rest/target/metron-rest-0.6.1.jar to /Users/scottcote/.m2/repository/org/apache/metron/metron-rest/0.6.1/metron-rest-0.6.1.jar
> [INFO] Installing /Users/scottcote/workspace/metron/metron-interface/metron-rest/pom.xml to /Users/scottcote/.m2/repository/org/apache/metron/metron-rest/0.6.1/metron-rest-0.6.1.pom
> [INFO] Installing /Users/scottcote/workspace/metron/metron-interface/metron-rest/target/metron-rest-0.6.1-archive.tar.gz to /Users/scottcote/.m2/repository/org/apache/metron/metron-rest/0.6.1/metron-rest-0.6.1-archive.tar.gz
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Metron ............................................. SUCCESS [ 14.439 s]
> [INFO] metron-stellar ..................................... SUCCESS [ 1.317 s]
> [INFO] stellar-common ..................................... SUCCESS [ 39.920 s]
> [INFO] metron-analytics ................................... SUCCESS [ 1.221 s]
> [INFO] metron-maas-common ................................. SUCCESS [ 1.142 s]
> [INFO] metron-platform .................................... SUCCESS [ 1.666 s]
> [INFO] metron-zookeeper ................................... SUCCESS [ 5.004 s]
> [INFO] metron-test-utilities .............................. SUCCESS [ 2.454 s]
> [INFO] metron-integration-test ............................ SUCCESS [ 20.909 s]
> [INFO] metron-maas-service ................................ SUCCESS [ 20.476 s]
> [INFO] metron-common ...................................... SUCCESS [ 34.899 s]
> [INFO] metron-statistics .................................. SUCCESS [ 33.713 s]
> [INFO] metron-writer ...................................... SUCCESS [ 32.067 s]
> [INFO] metron-storm-kafka-override ........................ SUCCESS [ 11.703 s]
> [INFO] metron-storm-kafka ................................. SUCCESS [ 2.069 s]
> [INFO] metron-hbase ....................................... SUCCESS [ 3.316 s]
> [INFO] metron-profiler-common ............................. SUCCESS [ 2.298 s]
> [INFO] metron-profiler-client ............................. SUCCESS [ 31.861 s]
> [INFO] metron-profiler-storm .............................. SUCCESS [01:24 min]
> [INFO] metron-profiler-spark .............................. SUCCESS [01:26 min]
> [INFO] metron-profiler-repl ............................... SUCCESS [ 22.633 s]
> [INFO] metron-hbase-client ................................ SUCCESS [ 23.168 s]
> [INFO] metron-enrichment .................................. SUCCESS [01:27 min]
> [INFO] metron-indexing .................................... SUCCESS [ 23.352 s]
> [INFO] metron-solr ........................................ SUCCESS [01:06 min]
> [INFO] metron-job ......................................... SUCCESS [ 1.818 s]
> [INFO] metron-pcap ........................................ SUCCESS [ 4.094 s]
> [INFO] metron-parsers ..................................... SUCCESS [01:14 min]
> [INFO] metron-pcap-backend ................................ SUCCESS [ 54.775 s]
> [INFO] metron-data-management ............................. SUCCESS [01:13 min]
> [INFO] metron-management .................................. SUCCESS [ 12.919 s]
> [INFO] elasticsearch-shaded ............................... SUCCESS [ 25.123 s]
> [INFO] metron-elasticsearch ............................... SUCCESS [01:22 min]
> [INFO] metron-deployment .................................. SUCCESS [ 1.265 s]
> [INFO] Elasticsearch Ambari Management Pack ............... SUCCESS [ 2.529 s]
> [INFO] Metron Ambari Management Pack ...................... SUCCESS [ 3.869 s]
> [INFO] metron-contrib ..................................... SUCCESS [ 1.151 s]
> [INFO] metron-docker ...................................... SUCCESS [ 4.692 s]
> [INFO] metron-performance ................................. SUCCESS [ 4.457 s]
> [INFO] metron-interface ................................... SUCCESS [ 9.772 s]
> [INFO] metron-config ...................................... SUCCESS [05:00 min]
> [INFO] metron-alerts ...................................... SUCCESS [03:48 min]
> [INFO] metron-rest-client ................................. SUCCESS [ 1.148 s]
> [INFO] metron-rest ........................................ SUCCESS [ 33.988 s]
> [INFO] site-book .......................................... SUCCESS [ 1.929 s]
> [INFO] 3rd party Functions (just for tests) ............... SUCCESS [ 0.583 s]
> [INFO] stellar-zeppelin ................................... SUCCESS [ 2.079 s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 05:46 min (Wall Clock)
> [INFO] Finished at: 2018-10-23T09:22:06-05:00
> [INFO] Final Memory: 220M/2041M
> [INFO] ------------------------------------------------------------------------
> ~/workspace/metron (master) $ cd metron-deployment/development/centos6/
> ~/workspace/metron/metron-deployment/development/centos6 (master) $ vagrant up
> Running with ansible-skip-tags: ["sensors"]
> Bringing machine 'node1' up with 'virtualbox' provider...
> ==> node1: Importing base box 'centos/6'...
> ==> node1: Matching MAC address for NAT networking...
> ==> node1: Checking if box 'centos/6' is up to date...
> ==> node1: Setting the name of the VM: centos6_node1_1540304760857_3427
> ==> node1: Clearing any previously set network interfaces...
> ==> node1: Preparing network interfaces based on configuration...
> node1: Adapter 1: nat
> node1: Adapter 2: hostonly
> ==> node1: Forwarding ports...
> node1: 22 (guest) => 2222 (host) (adapter 1)
> ==> node1: Running 'pre-boot' VM customizations...
> ==> node1: Booting VM...
> ==> node1: Waiting for machine to boot. This may take a few minutes...
> node1: SSH address: 127.0.0.1:2222 <http://127.0.0.1:2222/>
> node1: SSH username: vagrant
> node1: SSH auth method: private key
> node1:
> node1: Vagrant insecure key detected. Vagrant will automatically replace
> node1: this with a newly generated keypair for better security.
> node1:
> node1: Inserting generated public key within guest...
> node1: Removing insecure key from the guest if it's present...
> node1: Key inserted! Disconnecting and reconnecting using new SSH key...
> ==> node1: Machine booted and ready!
> ==> node1: Checking for guest additions in VM...
> node1: No guest additions were detected on the base box for this VM! Guest
> node1: additions are required for forwarded ports, shared folders, host only
> node1: networking, and more. If SSH fails on this machine, please install
> node1: the guest additions and repackage the box to continue.
> node1:
> node1: This is not an error message; everything may continue to work properly,
> node1: in which case you may ignore this message.
> ==> node1: Setting hostname...
> ==> node1: Configuring and enabling network interfaces...
> node1: SSH address: 127.0.0.1:2222 <http://127.0.0.1:2222/>
> node1: SSH username: vagrant
> node1: SSH auth method: private key
> ==> node1: Rsyncing folder: /Users/scottcote/workspace/metron/metron-deployment/development/centos6/ => /vagrant
> ==> node1: [vagrant-hostmanager:guests] Updating hosts file on active guest virtual machines...
> ==> node1: [vagrant-hostmanager:host] Updating hosts file on your workstation (password may be required)...
> Password:
> ==> node1: Running provisioner: ansible...
> Vagrant has automatically selected the compatibility mode '2.0'
> according to the Ansible version installed (2.2.2.0).
> Alternatively, the compatibility mode can be specified in your Vagrantfile:
> https://www.vagrantup.com/docs/provisioning/ansible_common.html#compatibility_mode <https://www.vagrantup.com/docs/provisioning/ansible_common.html#compatibility_mode>
> node1: Running ansible-playbook...
> PYTHONUNBUFFERED=1 ANSIBLE_FORCE_COLOR=true ANSIBLE_HOST_KEY_CHECKING=false ANSIBLE_SSH_ARGS='-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o IdentityFile=/Users/scottcote/workspace/metron/metron-deployment/development/centos6/.vagrant/machines/node1/virtualbox/private_key -o ControlMaster=auto -o ControlPersist=60s' ansible-playbook --connection=ssh --timeout=30 --extra-vars=ansible_user\=\'vagrant\' --limit="node1" --inventory-file=ansible/inventory --become -vvv --skip-tags=sensors ansible/playbook.yml
> Using /Users/scottcote/workspace/metron/metron-deployment/development/centos6/ansible.cfg as config file
> ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.
> The error appears to have been in '/Users/scottcote/workspace/metron/metron-deployment/ansible/roles/enable-swap/tasks/main.yml': line 29, column 3, but may
> be elsewhere in the file depending on the exact syntax problem.
> The offending line appears to be:
> - include_tasks: enable-swap.yml
> ^ here
> The error appears to have been in '/Users/scottcote/workspace/metron/metron-deployment/ansible/roles/enable-swap/tasks/main.yml': line 29, column 3, but may
> be elsewhere in the file depending on the exact syntax problem.
> The offending line appears to be:
> - include_tasks: enable-swap.yml
> ^ here
> Ansible failed to complete successfully. Any error output should be
> visible above. Please fix these errors and try again.
>
> 5. enable solr and disable elastic
>
> go to `ansible/inventory/group_vars/all`
>
> and set
> `# search
> install_elasticsearch: False
> install_solr: True
> solr_collection_name: Metron
> solr_number_shards: 1
> solr_replication_factor: 1
> elasticsearch_transport_port: 9300
> ## The elasticsearch_network_interface must be in the form specified for Elasticsearch, with leading and trailing underscores.
> elasticsearch_network_interface: _eth1_
> elasticsearch_web_port: 9200
> ansible/inventory/group_vars/all
> `
>
> 6 make install verbose
>
> mod this file ~/devprojects/metron/metron-deployment/development/centos6/Vagrantfile
>
> er, metron/metron-deployment/development/centos6/Vagrantfile
> under config.vm.provision add this
> ansible.verbose = "vvv"
>
>
> 7. ansible correction based on feedback that I'm using head of master and its not in synch with the specified version of ansible on the page
>
>
> `pip install ansible==2.5.0`
>
> then do `vagrant up`
>
Re: Building Metron with Vagrant from Master Branch on Mac OS X using SOLR
Posted by Michael Miklavcic <mi...@gmail.com>.
Thanks for the feedback Scott. I see 3 categories:
1. Some setup and prereq doc clarification for newcomers, e.g. need for
xcode install, clarify doc version compatibility
2. Added details for configuring Solr instead of ES
3. Some added details around self-help on debugging the install, e.g.
enabling vvv with ansible (I was actually the dev that worked through that
with Ward), some suggestion about finding errors in Ambari.
Some of this could probably be simple Ansible enhancements. Have I missed
anything? Are you still in the middle of getting full dev up or is it
working for you now?
Best,
Mike Miklavcic
On Oct 24, 2018 4:03 PM, "Scott Cote" <sc...@lucidworks.com> wrote:
Hello Fellow Metronites:
As I explored my very first setup of Metron for the vagrant install, I ran
into a few hiccups.
I have listed below a summary of a rather long email with the details
broken out. Apologies for the lengthiness.
1. Hiccups while performing the pip install ansible step
2. install Xcode command line tools
3. install Xcode command line tools
4. After steps 1 and 2 were performed, I did a build from master and
attempted to startup the vagrant instance. Below is a dump of that
attempts. Vagrant still failed to come up. More fixes:
5. enable solr and disable elastic
6. make install verbose
7. ansible correction based on feedback that I'm using head of master and
its not in synch with the specified version of ansible on the page
1 Hiccups while performing the pip install ansible step:
```I was able to get this to work by upgrading openssl using brew
brew upgrade openssl
and then setting the CPPFLAGS and LDFLAGS to point to the openssl lib from
brew
export CPPFLAGS=-I/usr/local/opt/openssl/include
export LDFLAGS=-L/usr/local/opt/openssl/lib```
when installing Vagrant at the step `pip install ansible==2.2.2.0`
i encountered problems with the crypt... python packages.
had to ensure that `brew upgrade openssl`
was performed.
2. install Xcode command line tools (again for some reason???)
`$ xcode-select --install`
3. errors that occurred while sensible installed with bad SSL brew casks. -
(see step 1):
```creating
build/temp.macosx-10.13-intel-2.7/build/temp.macosx-10.13-intel-2.7
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g
-Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX
-DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv
-Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe
-I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
-c build/temp.macosx-10.13-intel-2.7/_openssl.c -o
build/temp.macosx-10.13-intel-2.7/build/temp.macosx-10.13-intel-2.7/_openssl.o
-Wconversion -Wno-error=sign-conversion
build/temp.macosx-10.13-intel-2.7/_openssl.c:498:10: fatal error:
'openssl/opensslv.h' file not found
#include <openssl/opensslv.h>
^~~~~~~~~~~~~~~~~~~~
1 error generated.
error: command 'cc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python -c "import setuptools,
tokenize;__file__='/private/var/folders/vx/n3g3g7k94lg8pxz1hdp4bhb40000gp/T/pip-build-uWUxRG/cryptography/setup.py';exec(compile(getattr(tokenize,
'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))"
install --record
/var/folders/vx/n3g3g7k94lg8pxz1hdp4bhb40000gp/T/pip-OpX1S3-record/install-record.txt
--single-version-externally-managed --compile" failed with error code 1 in
/private/var/folders/vx/n3g3g7k94lg8pxz1hdp4bhb40000gp/T/pip-build-uWUxRG/cryptography```
4. After steps 1 and 2 were performed, I did a build from master and
attempted to startup the vagrant instance. Below is a dump of that
attempts. Vagrant still failed to come up. More fixes:
Tried to just use vagrant up and got a fail ....
This snippet has the following pieces of information to help you help me:
4.1. information about my machine/env
4.2. overview of sequence of commands to reproduce
4.3. terminal dump
Here are the details:
4.1. Information about my machine
4.1.1. unamme output:
~/workspace/metron/metron-deployment/development/centos6 (master) $ uname -a
Darwin ScottCotesLWMBP.fios-router.home 17.7.0 Darwin Kernel Version
17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64
x86_64
4.1.2. platform-info.sh output
~/workspace/metron/metron-deployment/scripts (master) $ ./platform-info.sh
Metron 0.6.1
--
* master
--
commit 0ba18026222f004316b96633780509c71934121e (HEAD -> master,
origin/master, origin/HEAD)
Author: JonZeolla <ze...@gmail.com>
Date: Mon Oct 22 19:48:56 2018 -0400
METRON-1826 Update librdkafka and devtoolset (JonZeolla via jonzeolla)
closes apache/metron#1238
--
metron-deployment/development/centos6/Vagrantfile | 1
+
metron-deployment/development/centos6/ansible/inventory/group_vars/all | 4
++--
2 files changed, 3 insertions(+), 2 deletions(-)
--
ansible 2.2.2.0
config file =
configured module search path = Default w/o overrides
--
Vagrant 2.1.5
--
vagrant-hostmanager (1.8.9, global)
--
Python 2.7.10
--
Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426;
2017-04-03T14:39:06-05:00)
Maven home: /usr/local/Cellar/maven/3.5.0/libexec
Java version: 1.8.0_181, vendor: Oracle Corporation
Java home:
/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac"
--
Docker version 18.06.1-ce, build e68fc7a
--
node
v10.7.0
--
npm
6.2.0
--
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr
--with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 10.0.0 (clang-1000.11.45.2)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
--
Compiler is C++11 compliant
--
Darwin ScottCotesLWMBP.fios-router.home 17.7.0 Darwin Kernel Version
17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64
x86_64
--
Total System Memory = 16384 MB
Processor Model: Intel(R) Core(TM) i7-4770HQ CPU
Processor Speed: 2.20GHz
Total Physical Processors: 4
Total cores: 4
Disk information:
/dev/disk1s1 466Gi 408Gi 50Gi 90% 3427183 9223372036851348624
0% /
/dev/disk1s4 466Gi 7.0Gi 50Gi 13% 7 9223372036854775800
0% /private/var/vm
This CPU appears to support virtualization
4.2. overview of sequence of commands to reproduce
followed instructions for vagrant setup with the last following steps:
4.2.1. ~/workspace/metron/metron-deployment/development/centos6 (master) $
vagrant destroy -f
4.2.2. ~/workspace/metron/metron-deployment/development/centos6 (master) $
cd ../../..
4.2.3. ~/workspace/metron (master) $ git pull
4.2.4. ~/workspace/metron (master) $ mvn clean install -DskipTests -T C4
4.2.5. ~/workspace/metron (master) $ cd
metron-deployment/development/centos6
4.2.4. ~/workspace/metron/metron-deployment/development/centos6 (master) $
vagrant up
4.3. terminal dump
...................................................
..--->>>redacted terminal dump for brevity<<<--- ..
...................................................
[INFO] --- maven-install-plugin:2.4:install (default-install) @ metron-rest
---
[INFO] Installing
/Users/scottcote/workspace/metron/metron-interface/metron-rest/target/metron-rest-0.6.1.jar
to
/Users/scottcote/.m2/repository/org/apache/metron/metron-rest/0.6.1/metron-rest-0.6.1.jar
[INFO] Installing
/Users/scottcote/workspace/metron/metron-interface/metron-rest/pom.xml to
/Users/scottcote/.m2/repository/org/apache/metron/metron-rest/0.6.1/metron-rest-0.6.1.pom
[INFO] Installing
/Users/scottcote/workspace/metron/metron-interface/metron-rest/target/metron-rest-0.6.1-archive.tar.gz
to
/Users/scottcote/.m2/repository/org/apache/metron/metron-rest/0.6.1/metron-rest-0.6.1-archive.tar.gz
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Metron ............................................. SUCCESS [
14.439 s]
[INFO] metron-stellar ..................................... SUCCESS [
1.317 s]
[INFO] stellar-common ..................................... SUCCESS [
39.920 s]
[INFO] metron-analytics ................................... SUCCESS [
1.221 s]
[INFO] metron-maas-common ................................. SUCCESS [
1.142 s]
[INFO] metron-platform .................................... SUCCESS [
1.666 s]
[INFO] metron-zookeeper ................................... SUCCESS [
5.004 s]
[INFO] metron-test-utilities .............................. SUCCESS [
2.454 s]
[INFO] metron-integration-test ............................ SUCCESS [
20.909 s]
[INFO] metron-maas-service ................................ SUCCESS [
20.476 s]
[INFO] metron-common ...................................... SUCCESS [
34.899 s]
[INFO] metron-statistics .................................. SUCCESS [
33.713 s]
[INFO] metron-writer ...................................... SUCCESS [
32.067 s]
[INFO] metron-storm-kafka-override ........................ SUCCESS [
11.703 s]
[INFO] metron-storm-kafka ................................. SUCCESS [
2.069 s]
[INFO] metron-hbase ....................................... SUCCESS [
3.316 s]
[INFO] metron-profiler-common ............................. SUCCESS [
2.298 s]
[INFO] metron-profiler-client ............................. SUCCESS [
31.861 s]
[INFO] metron-profiler-storm .............................. SUCCESS [01:24
min]
[INFO] metron-profiler-spark .............................. SUCCESS [01:26
min]
[INFO] metron-profiler-repl ............................... SUCCESS [
22.633 s]
[INFO] metron-hbase-client ................................ SUCCESS [
23.168 s]
[INFO] metron-enrichment .................................. SUCCESS [01:27
min]
[INFO] metron-indexing .................................... SUCCESS [
23.352 s]
[INFO] metron-solr ........................................ SUCCESS [01:06
min]
[INFO] metron-job ......................................... SUCCESS [
1.818 s]
[INFO] metron-pcap ........................................ SUCCESS [
4.094 s]
[INFO] metron-parsers ..................................... SUCCESS [01:14
min]
[INFO] metron-pcap-backend ................................ SUCCESS [
54.775 s]
[INFO] metron-data-management ............................. SUCCESS [01:13
min]
[INFO] metron-management .................................. SUCCESS [
12.919 s]
[INFO] elasticsearch-shaded ............................... SUCCESS [
25.123 s]
[INFO] metron-elasticsearch ............................... SUCCESS [01:22
min]
[INFO] metron-deployment .................................. SUCCESS [
1.265 s]
[INFO] Elasticsearch Ambari Management Pack ............... SUCCESS [
2.529 s]
[INFO] Metron Ambari Management Pack ...................... SUCCESS [
3.869 s]
[INFO] metron-contrib ..................................... SUCCESS [
1.151 s]
[INFO] metron-docker ...................................... SUCCESS [
4.692 s]
[INFO] metron-performance ................................. SUCCESS [
4.457 s]
[INFO] metron-interface ................................... SUCCESS [
9.772 s]
[INFO] metron-config ...................................... SUCCESS [05:00
min]
[INFO] metron-alerts ...................................... SUCCESS [03:48
min]
[INFO] metron-rest-client ................................. SUCCESS [
1.148 s]
[INFO] metron-rest ........................................ SUCCESS [
33.988 s]
[INFO] site-book .......................................... SUCCESS [
1.929 s]
[INFO] 3rd party Functions (just for tests) ............... SUCCESS [
0.583 s]
[INFO] stellar-zeppelin ................................... SUCCESS [
2.079 s]
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 05:46 min (Wall Clock)
[INFO] Finished at: 2018-10-23T09:22:06-05:00
[INFO] Final Memory: 220M/2041M
[INFO]
------------------------------------------------------------------------
~/workspace/metron (master) $ cd metron-deployment/development/centos6/
~/workspace/metron/metron-deployment/development/centos6 (master) $ vagrant
up
Running with ansible-skip-tags: ["sensors"]
Bringing machine 'node1' up with 'virtualbox' provider...
==> node1: Importing base box 'centos/6'...
==> node1: Matching MAC address for NAT networking...
==> node1: Checking if box 'centos/6' is up to date...
==> node1: Setting the name of the VM: centos6_node1_1540304760857_3427
==> node1: Clearing any previously set network interfaces...
==> node1: Preparing network interfaces based on configuration...
node1: Adapter 1: nat
node1: Adapter 2: hostonly
==> node1: Forwarding ports...
node1: 22 (guest) => 2222 (host) (adapter 1)
==> node1: Running 'pre-boot' VM customizations...
==> node1: Booting VM...
==> node1: Waiting for machine to boot. This may take a few minutes...
node1: SSH address: 127.0.0.1:2222
node1: SSH username: vagrant
node1: SSH auth method: private key
node1:
node1: Vagrant insecure key detected. Vagrant will automatically replace
node1: this with a newly generated keypair for better security.
node1:
node1: Inserting generated public key within guest...
node1: Removing insecure key from the guest if it's present...
node1: Key inserted! Disconnecting and reconnecting using new SSH key...
==> node1: Machine booted and ready!
==> node1: Checking for guest additions in VM...
node1: No guest additions were detected on the base box for this VM!
Guest
node1: additions are required for forwarded ports, shared folders, host
only
node1: networking, and more. If SSH fails on this machine, please
install
node1: the guest additions and repackage the box to continue.
node1:
node1: This is not an error message; everything may continue to work
properly,
node1: in which case you may ignore this message.
==> node1: Setting hostname...
==> node1: Configuring and enabling network interfaces...
node1: SSH address: 127.0.0.1:2222
node1: SSH username: vagrant
node1: SSH auth method: private key
==> node1: Rsyncing folder:
/Users/scottcote/workspace/metron/metron-deployment/development/centos6/ =>
/vagrant
==> node1: [vagrant-hostmanager:guests] Updating hosts file on active guest
virtual machines...
==> node1: [vagrant-hostmanager:host] Updating hosts file on your
workstation (password may be required)...
Password:
==> node1: Running provisioner: ansible...
Vagrant has automatically selected the compatibility mode '2.0'
according to the Ansible version installed (2.2.2.0).
Alternatively, the compatibility mode can be specified in your Vagrantfile:
https://www.vagrantup.com/docs/provisioning/ansible_common.html#compatibility_mode
node1: Running ansible-playbook...
PYTHONUNBUFFERED=1 ANSIBLE_FORCE_COLOR=true ANSIBLE_HOST_KEY_CHECKING=false
ANSIBLE_SSH_ARGS='-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o
IdentityFile=/Users/scottcote/workspace/metron/metron-deployment/development/centos6/.vagrant/machines/node1/virtualbox/private_key
-o ControlMaster=auto -o ControlPersist=60s' ansible-playbook
--connection=ssh --timeout=30 --extra-vars=ansible_user\=\'vagrant\'
--limit="node1" --inventory-file=ansible/inventory --become -vvv
--skip-tags=sensors ansible/playbook.yml
Using
/Users/scottcote/workspace/metron/metron-deployment/development/centos6/ansible.cfg
as config file
ERROR! no action detected in task. This often indicates a misspelled module
name, or incorrect module path.
The error appears to have been in
'/Users/scottcote/workspace/metron/metron-deployment/ansible/roles/enable-swap/tasks/main.yml':
line 29, column 3, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- include_tasks: enable-swap.yml
^ here
The error appears to have been in
'/Users/scottcote/workspace/metron/metron-deployment/ansible/roles/enable-swap/tasks/main.yml':
line 29, column 3, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- include_tasks: enable-swap.yml
^ here
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.
5. enable solr and disable elastic
go to `ansible/inventory/group_vars/all`
and set
`# search
install_elasticsearch: False
install_solr: True
solr_collection_name: Metron
solr_number_shards: 1
solr_replication_factor: 1
elasticsearch_transport_port: 9300
## The elasticsearch_network_interface must be in the form specified for
Elasticsearch, with leading and trailing underscores.
elasticsearch_network_interface: _eth1_
elasticsearch_web_port: 9200
ansible/inventory/group_vars/all
`
6 make install verbose
mod this file
~/devprojects/metron/metron-deployment/development/centos6/Vagrantfile
er, metron/metron-deployment/development/centos6/Vagrantfile
under config.vm.provision add this
ansible.verbose = "vvv"
7. ansible correction based on feedback that I'm using head of master and
its not in synch with the specified version of ansible on the page
`pip install ansible==2.5.0`
then do `vagrant up`