You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@metron.apache.org by ot...@apache.org on 2017/11/28 13:17:49 UTC
metron git commit: METRON-1333 Ansible-Docker can no longer build
metron (ottobackwards) closes apache/metron#848
Repository: metron
Updated Branches:
refs/heads/master d07833a25 -> 0f8a323dd
METRON-1333 Ansible-Docker can no longer build metron (ottobackwards) closes apache/metron#848
Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/0f8a323d
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/0f8a323d
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/0f8a323d
Branch: refs/heads/master
Commit: 0f8a323ddabf93617da514ccba66ea8a408893bd
Parents: d07833a
Author: ottobackwards <ot...@gmail.com>
Authored: Tue Nov 28 06:49:13 2017 -0500
Committer: otto <ot...@apache.org>
Committed: Tue Nov 28 06:49:13 2017 -0500
----------------------------------------------------------------------
README.md | 6 +++
.../packaging/docker/ansible-docker/Dockerfile | 19 +++++++++-
.../packaging/docker/ansible-docker/README.md | 39 ++++++++++++++++----
3 files changed, 54 insertions(+), 10 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/metron/blob/0f8a323d/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index f3b2765..91bef3d 100644
--- a/README.md
+++ b/README.md
@@ -87,6 +87,7 @@ $ mvn clean install -PHDP-2.5.0.0
You can swap "install" for "package" in the commands above if you don't want to deploy the artifacts to your local .m2 repo.
+
# Build Metron Reporting
To build and run reporting with code coverage:
@@ -102,6 +103,11 @@ $ mvn clean install -DskipTests site site:stage-deploy site:deploy
The staged site is deployed to /tmp/metron/site/index.html, and can be viewed by opening the file in a browser.
+## Building with Docker
+
+A Docker container with all the required software, with the proper versions, is available to be used as well.
+see [ansible-docker](metron-deployment/packaging/docker/ansible-docker)
+
# Navigating the Architecture
Metron is at its core a Kappa architecture with Apache Storm as the processing
http://git-wip-us.apache.org/repos/asf/metron/blob/0f8a323d/metron-deployment/packaging/docker/ansible-docker/Dockerfile
----------------------------------------------------------------------
diff --git a/metron-deployment/packaging/docker/ansible-docker/Dockerfile b/metron-deployment/packaging/docker/ansible-docker/Dockerfile
index 3fc8de5..ebf002a 100644
--- a/metron-deployment/packaging/docker/ansible-docker/Dockerfile
+++ b/metron-deployment/packaging/docker/ansible-docker/Dockerfile
@@ -14,13 +14,18 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-FROM centos:centos6
+FROM centos:centos6.9
MAINTAINER Apache Metron
RUN yum install -y tar
RUN yum install -y wget
+# base development tools required
RUN yum groupinstall -y "Development tools"
+# newer cpp 11 support required for building node modules
+RUN yum install -y centos-release-scl
+RUN yum install -y devtoolset-4-gcc-c++ devtoolset-4-gcc
RUN yum install -y zlib-dev openssl-devel sqlite-devel bzip2-devel libffi-devel
+# install python 2.7.11 but do not make it the default
RUN wget https://www.python.org/ftp/python/2.7.11/Python-2.7.11.tgz -O /usr/src/Python-2.7.11.tgz
WORKDIR /usr/src
RUN tar xvf Python-2.7.11.tgz
@@ -33,18 +38,28 @@ RUN tar xvf setuptools-11.3.tar.gz
WORKDIR /usr/src/setuptools-11.3
RUN python2.7 setup.py install
RUN easy_install-2.7 pip
+# install ansible and set the configuration var
RUN pip2.7 install ansible==2.0.0.2
RUN pip2.7 install boto
COPY ansible.cfg /root/
ENV ANSIBLE_CONFIG /root/ansible.cfg
+# java
RUN yum install -y java-1.8.0-openjdk java-1.8.0-openjdk-devel
RUN yum install -y which
RUN yum install -y nss
WORKDIR /usr/src
+# setup maven
RUN wget http://apache.cs.utah.edu/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
RUN tar xzvf apache-maven-3.3.9-bin.tar.gz
RUN mv apache-maven-3.3.9 /opt/maven
RUN ln -s /opt/maven/bin/mvn /usr/bin/mvn
-RUN yum -y install asciidoc rpm-build rpm2cpio tar unzip xmlto zip rpmlint && yum clean all
+# install rpm tools required to build rpms
+RUN yum -y install asciidoc rpm-build rpm2cpio tar unzip xmlto zip rpmlint make && yum clean all
+# create a .bashrc for root, enabling the cpp 11 toolset
+RUN touch /root/.bashrc \
+ && cat '/opt/rh/devtoolset-4/enable' >> /root/.bashrc
+# install node so that the node dependencies can be packaged into the RPMs
+RUN curl --silent --location https://rpm.nodesource.com/setup_6.x | bash -
+RUN yum -y install nodejs
WORKDIR /root
http://git-wip-us.apache.org/repos/asf/metron/blob/0f8a323d/metron-deployment/packaging/docker/ansible-docker/README.md
----------------------------------------------------------------------
diff --git a/metron-deployment/packaging/docker/ansible-docker/README.md b/metron-deployment/packaging/docker/ansible-docker/README.md
index 3d50d23..0c41d10 100644
--- a/metron-deployment/packaging/docker/ansible-docker/README.md
+++ b/metron-deployment/packaging/docker/ansible-docker/README.md
@@ -1,17 +1,40 @@
# Overview
-The Metron ansible-docker container is provided in an effort reduce the installation burden of deploying Metron in a live envirionment.
-It is provisioned with software required to sucessfully run the deployment scripts.
+The Metron ansible-docker container is provided in an effort reduce the installation burden of building Metron.
+It may also be used to deploy Metron in a live environment.
+It is provisioned with software required to sucessfully build metron and run the deployment scripts.
## Building the Container
1. Install Docker ( https://www.docker.com/products/overview )
2. Navigate to \<project-directory\>/metron-deployment/packaging/docker/ansible-docker
3. Build the container `docker build -t ansible-docker:2.0.0.2 .`
-## Using the Container
-Full instructions are found on the wiki at https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65144361
+## Using the Container to build metron
+anytime after building the container you can run it with the following command
+
+`docker run -it -v \<project-directory\>:/root/metron ansible-docker:2.0.0.2 bash`
+
+If you are going to build metron multiple times, you may want to map the /root/.m2 maven
+repo from outside of the container so that you don't start with an empty repo every build and have to download
+the world.
+
+`docker run -it -v \<project-directory\>:/root/metron -v \<your .m2 directory\>:/root/.m2 ansible-docker:2.0.0.2 bash`
+
+After running the container:
+
+1. cd /root/metron
+2. run build commands, for example:
+ - build metron without tests : `mvn clean package -DskipTests`
+ - build metron and build the rpms as well : `mvn clean install && cd metron-deployment && mvn package -P build-rpms`
+
+If you wish to use this build with a vagrant instance, then after building with rpms as above, modify
+your usual vagrant up command to skip the build role, as so:
-tl;dr:
+`vagrant --ansible-skip-tags="build,quick_dev" up`
+
+
+## Using the Container for deployment
+
+> Note these instructions are outdated
+
+Full instructions are found on the wiki at https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65144361
-1. docker run -it -v \<project-directory\>:/root/metron ansible-docker:2.0.0.2 bash
-2. cd /root/metron
-3. mvn clean package -DskipTests