You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2020/04/29 18:37:49 UTC
[GitHub] [accumulo-proxy] mjwall commented on a change in pull request #20: accumulo-proxy docker container
mjwall commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417524869
##########
File path: Dockerfile
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM openjdk:8
+
+EXPOSE 42424
+
+WORKDIR /opt/accumulo-proxy
+
+ARG HADOOP_VERSION=3.2.1
+ARG ZOOKEEPER_VERSION=3.5.7
+ARG ACCUMULO_VERSION=2.0.0
+ARG ACCUMULO_PROXY_VERSION=2.0.0-SNAPSHOT
+
+ARG HADOOP_HASH=a57962a24d178193349917730bf95cdc99bde9df
+ARG ZOOKEEPER_HASH=619928c8553b62775119e3d7d143a4714a160365
+ARG ACCUMULO_HASH=b72bf5c3dcaa25387933a032925046234f30e17a
+
+# Download from Apache mirrors instead of archive #9
+ENV APACHE_DIST_URLS \
+ https://www.apache.org/dyn/closer.cgi?action=download&filename= \
+# if the version is outdated (or we're grabbing the .asc file), we might have to pull from the dist/archive :/
+ https://www-us.apache.org/dist/ \
+ https://www.apache.org/dist/ \
+ https://archive.apache.org/dist/
+
+RUN set -eux; \
+ download_bin() { \
+ local f="$1"; shift; \
+ local hash="$1"; shift; \
+ local distFile="$1"; shift; \
+ local success=; \
+ local distUrl=; \
+ for distUrl in ${APACHE_DIST_URLS}; do \
+ if wget -nv -O "/tmp/${f}" "${distUrl}${distFile}"; then \
+ success=1; \
+ # Checksum the download
+ echo "${hash}" "/tmp/${f}" | sha1sum -c -; \
+ break; \
+ fi; \
+ done; \
+ [ -n "${success}" ]; \
+ };\
+ \
+ download_bin "apache-zookeeper.tar.gz" "${ZOOKEEPER_HASH}" "zookeeper/zookeeper-${ZOOKEEPER_VERSION}/apache-zookeeper-${ZOOKEEPER_VERSION}-bin.tar.gz"; \
+ download_bin "hadoop.tar.gz" "$HADOOP_HASH" "hadoop/core/hadoop-${HADOOP_VERSION}/hadoop-$HADOOP_VERSION.tar.gz"; \
+ download_bin "accumulo.tar.gz" "${ACCUMULO_HASH}" "accumulo/${ACCUMULO_VERSION}/accumulo-${ACCUMULO_VERSION}-bin.tar.gz";
+
+# Install the dependencies into /opt/
Review comment:
I don't know if it matters a whole lot, but you could combine the tar xzf lines with the above line and then remove the files you downloaded which would make the layer smaller. Maybe the size doesn't matter since you are not pushing anywhere
##########
File path: Dockerfile
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM openjdk:8
+
+EXPOSE 42424
+
+WORKDIR /opt/accumulo-proxy
+
+ARG HADOOP_VERSION=3.2.1
+ARG ZOOKEEPER_VERSION=3.5.7
+ARG ACCUMULO_VERSION=2.0.0
+ARG ACCUMULO_PROXY_VERSION=2.0.0-SNAPSHOT
+
+ARG HADOOP_HASH=a57962a24d178193349917730bf95cdc99bde9df
+ARG ZOOKEEPER_HASH=619928c8553b62775119e3d7d143a4714a160365
+ARG ACCUMULO_HASH=b72bf5c3dcaa25387933a032925046234f30e17a
+
+# Download from Apache mirrors instead of archive #9
+ENV APACHE_DIST_URLS \
+ https://www.apache.org/dyn/closer.cgi?action=download&filename= \
+# if the version is outdated (or we're grabbing the .asc file), we might have to pull from the dist/archive :/
+ https://www-us.apache.org/dist/ \
+ https://www.apache.org/dist/ \
+ https://archive.apache.org/dist/
+
+RUN set -eux; \
+ download_bin() { \
+ local f="$1"; shift; \
+ local hash="$1"; shift; \
+ local distFile="$1"; shift; \
+ local success=; \
+ local distUrl=; \
+ for distUrl in ${APACHE_DIST_URLS}; do \
+ if wget -nv -O "/tmp/${f}" "${distUrl}${distFile}"; then \
+ success=1; \
+ # Checksum the download
+ echo "${hash}" "/tmp/${f}" | sha1sum -c -; \
+ break; \
+ fi; \
+ done; \
+ [ -n "${success}" ]; \
+ };\
+ \
+ download_bin "apache-zookeeper.tar.gz" "${ZOOKEEPER_HASH}" "zookeeper/zookeeper-${ZOOKEEPER_VERSION}/apache-zookeeper-${ZOOKEEPER_VERSION}-bin.tar.gz"; \
+ download_bin "hadoop.tar.gz" "$HADOOP_HASH" "hadoop/core/hadoop-${HADOOP_VERSION}/hadoop-$HADOOP_VERSION.tar.gz"; \
+ download_bin "accumulo.tar.gz" "${ACCUMULO_HASH}" "accumulo/${ACCUMULO_VERSION}/accumulo-${ACCUMULO_VERSION}-bin.tar.gz";
+
+# Install the dependencies into /opt/
+RUN tar xzf /tmp/hadoop.tar.gz -C /opt/ && ln -s /opt/hadoop-${HADOOP_VERSION} /opt/hadoop
+RUN tar xzf /tmp/apache-zookeeper.tar.gz -C /opt/ && ln -s /opt/apache-zookeeper-${ZOOKEEPER_VERSION}-bin /opt/apache-zookeeper
+RUN tar xzf /tmp/accumulo.tar.gz -C /opt/ && ln -s /opt/accumulo-${ACCUMULO_VERSION} /opt/accumulo && sed -i 's/\${ZOOKEEPER_HOME}\/\*/\${ZOOKEEPER_HOME}\/\*\:\${ZOOKEEPER_HOME}\/lib\/\*/g' /opt/accumulo/conf/accumulo-env.sh
+
+ENV HADOOP_HOME /opt/hadoop
+ENV ZOOKEEPER_HOME /opt/apache-zookeeper
+ENV ACCUMULO_HOME /opt/accumulo
+
+# Add the proxy binary
+COPY target/accumulo-proxy-${ACCUMULO_PROXY_VERSION}-bin.tar.gz /tmp/
Review comment:
You need to run `mvn package` successfully first or this file will not be here. Not sure if there is a way to check and fail with a helpful message. Might want to mention that in the DOCKER.md as well.
##########
File path: DOCKER.md
##########
@@ -0,0 +1,122 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+This documentation covers how to stand up [accumulo-proxy](https://github.com/apache/accumulo-proxy/) within a Docker container.
+
+The guide covers:
+* Building the image
+* Configuring the `proxy.properties` file
+* Selecting an appropriate networking choice
+* Starting and stopping the container
+* Basic troubleshooting tips
+
+It is not recommended using this guide for a production instance of accumulo-proxy at this time.
+
+## Build the image using
+Invoke the docker build command to create a container image.
+```commandline
+docker build -t accumulo-proxy:latest .
+```
+
+## Default Configuration and Quickstart
+By default, the container image expects the following to be true:
+1. Your Accumulo instance name is "myinstance"
+2. Your ZooKeeper is available (and reachable from the container) at localhost:2181
+
+You can start the proxy using:
+```commandline
+docker run --rm -d -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest;
+```
+
+## Custom proxy.properties
+If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is:
+```commandline
+docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --network="host" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Networking configuration
+Container networking can be a very specialised subject therefore we document two common practices that should cover the majority of use cases for development.
+
+The proxy container must be able to access both Accumulo and ZooKeeper.
+
+The ZooKeeper location can be configured in the `conf/proxy.properties` file, so you can override this to an acceptable value (see "Custom proxy.properties" section)
+
+In order to communicate with Accumulo the container will need to be able to resolve the FQDN that the servers have registered in ZooKeeper. If using [fluo-uno](https://github.com/apache/fluo-uno) this is very likely the hostname of your development environment. We'll call this my.host.com and IP 192.168.0.1 for the rest of this document.
+
+### Host networking
+
+Host networking is the simplest mechanism but generally only works for linux hosts where docker has been installed on 'bare metal' e.g. through an RPM.
+
+You can test if this will work for you by executing the following:
+```commandline
+# Start the accumulo-proxy container and enter it
+docker run -it --rm -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest bash;
+
+# Install telnet and verify if you can connect to my.host.com:9995
+apt-get update && apt-get install telnet;
+telnet my.host.com 9995
Review comment:
Maybe mention this is one way to check connectivity. Could also use curl, something like `curl telnet://my.host.com:9995`. Perhaps give some sample output of what a correct connection looks like.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org