You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@fluo.apache.org by mw...@apache.org on 2017/09/11 15:05:09 UTC
[fluo-docker] branch master updated: Initial implementation of Fluo
Docker image (#1)
This is an automated email from the ASF dual-hosted git repository.
mwalch pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/fluo-docker.git
The following commit(s) were added to refs/heads/master by this push:
new fdeecf8 Initial implementation of Fluo Docker image (#1)
fdeecf8 is described below
commit fdeecf81f5512bffff09f4c48e62ea138b01fe7c
Author: Mike Walch <mw...@apache.org>
AuthorDate: Mon Sep 11 11:05:07 2017 -0400
Initial implementation of Fluo Docker image (#1)
* Depends on local fluo-1.2.0-SNAPSHOT tarball
---
.gitignore | 1 +
Dockerfile | 48 +++++++++++++++++++++++
README.md | 127 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
3 files changed, 175 insertions(+), 1 deletion(-)
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..335ec95
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1 @@
+*.tar.gz
diff --git a/Dockerfile b/Dockerfile
new file mode 100644
index 0000000..ab65e0e
--- /dev/null
+++ b/Dockerfile
@@ -0,0 +1,48 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM openjdk:8
+
+ARG HADOOP_VERSION
+ARG ZOOKEEPER_VERSION
+ARG ACCUMULO_VERSION
+
+ENV HADOOP_VERSION ${HADOOP_VERSION:-2.7.3}
+ENV ZOOKEEPER_VERSION ${ZOOKEEPER_VERSION:-3.4.9}
+ENV ACCUMULO_VERSION ${ACCUMULO_VERSION:-1.8.1}
+ENV FLUO_VERSION 1.2.0-SNAPSHOT
+
+RUN curl -sL http://archive.apache.org/dist/hadoop/core/hadoop-$HADOOP_VERSION/hadoop-$HADOOP_VERSION.tar.gz | tar -xzC /tmp
+RUN curl -sL http://archive.apache.org/dist/zookeeper/zookeeper-$ZOOKEEPER_VERSION/zookeeper-$ZOOKEEPER_VERSION.tar.gz | tar -xzC /tmp
+RUN curl -sL http://archive.apache.org/dist/accumulo/$ACCUMULO_VERSION/accumulo-$ACCUMULO_VERSION-bin.tar.gz | tar -xzC /tmp
+#RUN curl -sL http://archive.apache.org/dist/fluo/fluo/$FLUO_VERSION/fluo-$FLUO_VERSION-bin.tar.gz | tar -xzC /tmp
+# Comment out line above and remove line below in 1.2.0 is released
+ADD ./fluo-$FLUO_VERSION-bin.tar.gz /tmp/
+
+RUN mv /tmp/hadoop-$HADOOP_VERSION /opt/hadoop
+RUN mv /tmp/zookeeper-$ZOOKEEPER_VERSION /opt/zookeeper
+RUN mv /tmp/accumulo-$ACCUMULO_VERSION /opt/accumulo
+RUN mv /tmp/fluo-$FLUO_VERSION /opt/fluo
+
+ENV HADOOP_PREFIX /opt/hadoop
+ENV ZOOKEEPER_HOME /opt/zookeeper
+ENV ACCUMULO_HOME /opt/accumulo
+ENV FLUO_HOME /opt/fluo
+ENV PATH "$PATH:$FLUO_HOME/bin"
+
+RUN /opt/fluo/lib/fetch.sh extra
+
+ENTRYPOINT ["fluo"]
+CMD ["help"]
diff --git a/README.md b/README.md
index 9b3e163..d4c4319 100644
--- a/README.md
+++ b/README.md
@@ -1 +1,126 @@
-# Apache Fluo Docker Image
+# Fluo Docker Image
+
+## Obtain the Docker image
+
+To obtain the docker image created by this project, you can either pull it from DockerHub at
+`mikewalch/fluo` or build it yourself. To pull the image from DockerHub, run the command below:
+
+ sudo docker pull mikewalch/fluo
+
+While it is easier to pull from DockerHub, the image will default to the software versions below:
+
+| Software | Version |
+|-------------|----------------|
+| [Fluo] | 1.2.0-SNAPSHOT |
+| [Accumulo] | 1.8.1 |
+| [Hadoop] | 2.7.3 |
+| [Zookeeper] | 3.4.9 |
+
+If these versions do not match what is running on your cluster, you should consider building
+your own image with matching versions. However, Fluo must be 1.2+. Below are instructions for
+building an image:
+
+1. Clone the Fluo docker repo
+
+ git clone git@github.com:mikewalch/fluo-docker.git
+
+2. Until Fluo 1.2 is released, build a Fluo tarball distribution and copy it to the root
+ directory of the repo.
+
+ git clone git@github.com:apache/fluo.git
+ cd fluo/
+ mvn clean package
+ cp modules/distribution/target/fluo-1.2.0-SNAPSHOT-bin.tar.gz /path/to/fluo-docker/
+
+3. Build the default Fluo docker image using the command below.
+
+ cd /path/to/fluo-docker
+ docker build -t fluo .
+
+ Or build the Fluo docker image with specific versions of Hadoop, Zookeeper, etc using the command below:
+
+ docker build --build-arg ZOOKEEPER_VERSION=3.4.8 --build-arg ACCUMULO_VERSION=1.7.3 --build-arg HADOOP_VERSION=2.7.0 -t fluo .
+
+## Image basics
+
+The entrypoint for the Fluo docker image is the `fluo` script. While the primary use
+case for this image is to start an oracle or worker, you can run other commands in the
+`fluo` script to test out the image:
+
+```bash
+# No arguments prints Fluo command usage
+docker run mikewalch/fluo
+# Print Fluo version
+docker run mikewalch/fluo version
+# Print Fluo classpath
+docker run mikewalch/fluo classpath
+```
+
+# Run Fluo Applications using Docker
+
+Before starting a Fluo oracle and worker using the Fluo Docker image, [initialize your Fluo application][application].
+Next, choose a method below to run the oracle and worker(s) of your Fluo application. In the examples below, the Fluo
+application is named `myapp` and was initialized using a Zookeeper node on `zkhost`.
+
+## Docker engine
+
+Use the `docker` command to start local docker containers.
+
+1. Start a Fluo oracle
+
+ docker run -d --network="host" mikewalch/fluo oracle -a myapp -o fluo.connection.zookeepers=zkhost/fluo
+
+2. Start Fluo worker(s). Execute this command multiple times to start multiple workers.
+
+ docker run -d --network="host" mikewalch/fluo worker -a myapp -o fluo.connection.zookeepers=zkhost/fluo
+
+## Marathon
+
+Using the [Marathon] UI, you can create applications using JSON configuration.
+
+The JSON below can be used to start a Fluo oracle.
+
+```json
+{
+ "id": "myapp-fluo-oracle",
+ "cmd": "fluo oracle -a myapp -o fluo.connection.zookeepers=zkhost/fluo",
+ "cpus": 1,
+ "mem": 256,
+ "disk": 0,
+ "instances": 1,
+ "container": {
+ "docker": {
+ "image": "mikewalch/fluo",
+ "network": "HOST"
+ },
+ "type": "DOCKER"
+ }
+}
+```
+
+The JSON below can be used to start Fluo worker(s). Modify instances to start multiple workers.
+
+```json
+{
+ "id": "myapp-fluo-worker",
+ "cmd": "fluo worker -a myapp -o fluo.connection.zookeepers=zkhost/fluo",
+ "cpus": 1,
+ "mem": 512,
+ "disk": 0,
+ "instances": 1,
+ "container": {
+ "docker": {
+ "image": "mikewalch/fluo",
+ "network": "HOST"
+ },
+ "type": "DOCKER"
+ }
+}
+```
+
+[Fluo]: https://fluo.apache.org/
+[Accumulo]: https://accumulo.apache.org/
+[Hadoop]: https://hadoop.apache.org/
+[Zookeeper]: https://zookeeper.apache.org/
+[application]: https://github.com/apache/fluo/blob/master/docs/applications.md
+[Marathon]: https://mesosphere.github.io/marathon/
--
To stop receiving notification emails like this one, please contact
['"commits@fluo.apache.org" <co...@fluo.apache.org>'].