You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2020/04/22 15:03:08 UTC
[GitHub] [accumulo-proxy] volmasoft opened a new pull request #20: WIP: accumulo-proxy docker container
volmasoft opened a new pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20
### Overview
Note this is a work in progress, still quite a bit of work to go but I thought I'd expose some of the work early.
I've tried to take the best of a few worlds and tried to keep consistency with both the accumulo-docker repo (https://github.com/apache/accumulo-docker) and the Apache guides here: (https://github.com/docker-library/official-images)
This also assumes that your instance is called "myinstance" and your zookeeper is available at "localhost:2181" from the container, this is NOT the case on some configurations such as Docker for Mac, but I'e left these as defaults for now until we figure out "(2) Configuration" below.
Note, I have left some commands commented out on the Dockerfile currently that make life easier to debug, these will be stripped before they get merged finally.
Currently I have a large number of TODOs and questions for people's input, each is numbered so feel free to answer some or all of them with your thoughts :)
If no one has any strong feelings about some of the questions below then I'll just press on.
---
### (1) Repository
Given that accumulo has it's docker code in accumulo-docker do we need to do the same with this repo e.g. accumulo-proxy-docker or are we happy to merge it into the main code base?
---
### (2) Configuration
I've been debating at length how to allow the default configuration to be overiden.
My current thought is we should allow users to volume mount in (or rebuild the container) with a new proxy.properties and we could document both approaches, this would allow full configuration to be done and versioned outside of our accumulo-proxy project.
However we probably ought to make it easy to 'get started' and therefore we could also provide a further approach to overriding:
A) Allow people to provide an environment variable to override properties e.g. PROXY_CONFIG="key:pair" and having the Java code parse this after the default proxy.properties and update the properties - This would allow any property to be updated.
B) Allow people to only override 2 properties via environment variables (instance.name, instance.zookeepers) since these are more than likely to meet 99% of use cases, we can always add additional ones in the future.
My gut feeling is a mixture of the volume mounting/rebuild guide, plus option (B) is probably the right balance for now and we can always change these later on.
---
### (3) Software versions
Currently this change doesn't support selecting zookeeper/accumulo/hadoop versions, they're hard coded.
Question - Do we want to allow this support?
Question - Does this need doing on the initial commit or would it be acceptable to follow it up.
---
### (4) Automated Build
We could use one of the maven plugins to automatically create the docker container with a new profile, e.g. instead of `mvn package -Ptarball` we could provide a `mvn package -Pdocker` or similiar.
This would be different to the standard accumulo-docker repo which details how to build using the `docker build` command.
I'm happy either way.
---
### (5) Documentation in container
Looking at the accumulo-docker repo it was decided to put documentation into the container (README.md on [this line](https://github.com/apache/accumulo-docker/blob/master/Dockerfile#L38))
I'm not sure it provides value but I've added both the README.md and DOCKER.md into this container currently.
Should we strip this out?
---
### (6) CLASSPATH setting
When debugging this application I found that for some reason `accumulo classpath` provides a classpath which includes zookeeper, however it only includes (in my case) `/opt/apache-zookeeper/` and we actually need `/opt/apache-zookeeper/lib`.
Look at my diff `Dockerfile` and look for the line `ENV CLASSPATH=/opt/apache-zookeeper/lib/*`
Anyone know if I'm missing something in my configuration?
---
### (7) Install path shortcut
If you look at `Dockerfile` and look for the part where I add the dependencies to /opt/ (roughly line 61 starting with "# Install the dependencies into /opt/"
I take the versions of software and strip their first folder so that they are in folders similiar to /opt/hadoop/ in one step.
This may make the installed versions a bit less transparent if you were to jump into the container (e.g. it's not /opt/hadoop-3.2.1/)
Does anyone have any strong feelings about this?
--- COMMIT LOG ---
Initial commit of the work to stand up an accumulo-proxy inside a doker container.
I have only implemented support for Accumulo 2.x and by default this first commit contains:
* Accumulo 2.0.0
* Hadoop 3.2.1
* Zookeeper 3.5.7
A new document (DOCKER.md) has been created to start to document the implementation and usage guide which should allow others to test this if they so wished.
A number of outstanding questions will be posted on the issue, there is also a number of TODOs still required to be implemented that I'm tracking.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] keith-turner commented on a change in pull request #20: WIP: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
keith-turner commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417377216
##########
File path: DOCKER.md
##########
@@ -25,29 +25,86 @@ Invoke the docker build command to create a container image.
docker build -t accumulo-proxy:latest .
```
-## Default Configuration
+## Default Configuration and Quickstart
By default, the container image expects the following to be true:
-1. Your accumulo instance is named "myinstance"
+1. Your Accumulo instance name is "myinstance"
2. Your zookeeper is available (and reachable from the container) at localhost:2181
+You can start the proxy using:
+```commandline
+docker run --rm -d -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest;
+```
+
## Custom proxy.properties
If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is:
```commandline
-docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --add-host"FQDN:IP" --name accumulo-proxy accumulo-proxy:latest
+docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --network="host" --name accumulo-proxy accumulo-proxy:latest
```
-## Accumulo Instance Configuration
-In order for Thrift to communicate with the Accumulo instance you will need to provide the container with the FQDN and IP of the Accumulo instance and its relevant servers (e.g. tservers) to allow it to resolve the required DNS.
+## Networking configuration
+Container networking can be a very specialised subject therefore we document two common practices that should cover the majority of use cases for development.
+
+The proxy container must be able to access both Accumulo and Zookeeper.
+
+The Zookeeper location can be configured in the `conf/proxy.properties` file, so you can override this to an acceptable value (see "Custom proxy.properties" section)
+
+In order to communicate with Accumulo the container will need to be able to resolve the FQDN that the servers have registered in Zookeeper. If using [fluo-uno](https://github.com/apache/fluo-uno) this is very likely the hostname of your development environment. We'll call this my.host.com and IP 192.168.0.1 for the rest of this document.
+
+### Host networking
-If you are running an Accumulo instance with more than one tserver you should add each tserver's entry with a new `--add-host "FQDN:IP"` entry.
+Host networking is the simplest mechanism but generally only works for linux hosts where docker has been installed on 'bare metal' e.g. through an RPM.
+You can test if this will work for you by executing the following:
```commandline
-docker run --rm -d -p 42424:42424 --add-host "FQDN:IP" --name accumulo-proxy accumulo-proxy:latest
+# Start the accumulo-proxy container and enter it
+docker run -it --rm -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest bash;
+
+# Install telnet and verify if you can connect to my.host.com:9995
+apt-get update && apt-get install telnet;
+telnet my.host.com 9995
```
-If you run your Accumulo instance inside a container you can link the containers together using the legacy `--link` approach or place them in the same network ([see official docs](https://docs.docker.com/network/links/))
+If telnet can connect, then your container can resolve `my.host.com` correctly with host networking and therefore you can append `--network="host"` to your docker commands.
-Cleanup using:
+An example of using host networking:
+```commandline
+docker run --rm -d -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest
+```
+
+Note: You do not need to map your ports (-p) if using host networking, but we include it for clarity.
+
+For more details see the official docker documentation: [Use host Networking](https://docs.docker.com/network/host)
+
+### Non-Host networking
+
+If you run outside of a single node linux installation, e.g. Docker for Mac, Docker for Windows or use a VM to isolate your docker engine then you will likely need to take this path.
+
+Docker allows you to supply additional addresses to be resolved by the container, and these are automatically added by Docker to the /etc/hosts
+
+For each host add a `--add-host FQDN:IP` entry to your docker run command, you can add multiple entries if need to, see the official docs covering [network settings](https://docs.docker.com/engine/reference/run/#network-settings).
+
+An example of using this approach:
+
+```commandline
+docker run --rm -d -p 42424:42424 --add-host "my.host.com:192.168.0.1" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Cleanup
+Once completed you should stop and remove the container.
```commandline
docker stop accumulo-proxy;
+docker rm accumulo-proxy;
+```
+
+## Troubleshooting
+It can often be difficult to know where to start with troubleshooting inside containers, if you need to enter the container without starting the proxy we support this:
Review comment:
Could also link to the contact us page as another avenue for troubleshooting.
https://accumulo.apache.org/contact-us/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] volmasoft commented on a change in pull request #20: WIP: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
volmasoft commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417501989
##########
File path: DOCKER.md
##########
@@ -0,0 +1,114 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+
+A temporary guide on how to run this up in Docker.
Review comment:
Nice spot! I completely missed that I had left it as a temporary guide when I first committed the file. I've now added a better description.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] volmasoft commented on a change in pull request #20: WIP: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
volmasoft commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417502275
##########
File path: DOCKER.md
##########
@@ -0,0 +1,114 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+
+A temporary guide on how to run this up in Docker.
+
+## Build the image using
+Invoke the docker build command to create a container image.
+```commandline
+docker build -t accumulo-proxy:latest .
+```
+
+## Default Configuration and Quickstart
+By default, the container image expects the following to be true:
+1. Your Accumulo instance name is "myinstance"
+2. Your zookeeper is available (and reachable from the container) at localhost:2181
+
+You can start the proxy using:
+```commandline
+docker run --rm -d -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest;
+```
+
+## Custom proxy.properties
+If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is:
+```commandline
+docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --network="host" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Networking configuration
+Container networking can be a very specialised subject therefore we document two common practices that should cover the majority of use cases for development.
+
+The proxy container must be able to access both Accumulo and Zookeeper.
+
+The Zookeeper location can be configured in the `conf/proxy.properties` file, so you can override this to an acceptable value (see "Custom proxy.properties" section)
+
+In order to communicate with Accumulo the container will need to be able to resolve the FQDN that the servers have registered in Zookeeper. If using [fluo-uno](https://github.com/apache/fluo-uno) this is very likely the hostname of your development environment. We'll call this my.host.com and IP 192.168.0.1 for the rest of this document.
+
+### Host networking
+
+Host networking is the simplest mechanism but generally only works for linux hosts where docker has been installed on 'bare metal' e.g. through an RPM.
+
+You can test if this will work for you by executing the following:
+```commandline
+# Start the accumulo-proxy container and enter it
+docker run -it --rm -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest bash;
+
+# Install telnet and verify if you can connect to my.host.com:9995
+apt-get update && apt-get install telnet;
+telnet my.host.com 9995
+```
+
+If telnet can connect, then your container can resolve `my.host.com` correctly with host networking and therefore you can append `--network="host"` to your docker commands.
+
+An example of using host networking:
+```commandline
+docker run --rm -d -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest
+```
+
+Note: You do not need to map your ports (-p) if using host networking, but we include it for clarity.
+
+For more details see the official docker documentation: [Use host Networking](https://docs.docker.com/network/host)
+
+### Non-Host networking
+
+If you run outside of a single node linux installation, e.g. Docker for Mac, Docker for Windows or use a VM to isolate your docker engine then you will likely need to take this path.
+
+Docker allows you to supply additional addresses to be resolved by the container, and these are automatically added by Docker to the /etc/hosts
+
+For each host add a `--add-host FQDN:IP` entry to your docker run command, you can add multiple entries if need to, see the official docs covering [network settings](https://docs.docker.com/engine/reference/run/#network-settings).
+
+An example of using this approach:
+
+```commandline
+docker run --rm -d -p 42424:42424 --add-host "my.host.com:192.168.0.1" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Cleanup
+Once completed you should stop and remove the container.
+```commandline
+docker stop accumulo-proxy;
+docker rm accumulo-proxy;
+```
+
+## Troubleshooting
+It can often be difficult to know where to start with troubleshooting inside containers, if you need to enter the container without starting the proxy we support this:
+```commandline
+docker run -it --rm -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest bash
+```
+
+The container is very slim so if need be you can add additional tools using `apt`.
+
+If you wish to manually execute the accumulo-proxy in you can:
Review comment:
Fixed in latest commit along with other aspects.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] volmasoft commented on pull request #20: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
volmasoft commented on pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#issuecomment-621413968
Thanks for all the feedback 👍
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] volmasoft commented on a change in pull request #20: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
volmasoft commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417545571
##########
File path: Dockerfile
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM openjdk:8
+
+EXPOSE 42424
+
+WORKDIR /opt/accumulo-proxy
+
+ARG HADOOP_VERSION=3.2.1
+ARG ZOOKEEPER_VERSION=3.5.7
+ARG ACCUMULO_VERSION=2.0.0
+ARG ACCUMULO_PROXY_VERSION=2.0.0-SNAPSHOT
+
+ARG HADOOP_HASH=a57962a24d178193349917730bf95cdc99bde9df
+ARG ZOOKEEPER_HASH=619928c8553b62775119e3d7d143a4714a160365
+ARG ACCUMULO_HASH=b72bf5c3dcaa25387933a032925046234f30e17a
+
+# Download from Apache mirrors instead of archive #9
+ENV APACHE_DIST_URLS \
+ https://www.apache.org/dyn/closer.cgi?action=download&filename= \
+# if the version is outdated (or we're grabbing the .asc file), we might have to pull from the dist/archive :/
+ https://www-us.apache.org/dist/ \
+ https://www.apache.org/dist/ \
+ https://archive.apache.org/dist/
+
+RUN set -eux; \
+ download_bin() { \
+ local f="$1"; shift; \
+ local hash="$1"; shift; \
+ local distFile="$1"; shift; \
+ local success=; \
+ local distUrl=; \
+ for distUrl in ${APACHE_DIST_URLS}; do \
+ if wget -nv -O "/tmp/${f}" "${distUrl}${distFile}"; then \
+ success=1; \
+ # Checksum the download
+ echo "${hash}" "/tmp/${f}" | sha1sum -c -; \
+ break; \
+ fi; \
+ done; \
+ [ -n "${success}" ]; \
+ };\
+ \
+ download_bin "apache-zookeeper.tar.gz" "${ZOOKEEPER_HASH}" "zookeeper/zookeeper-${ZOOKEEPER_VERSION}/apache-zookeeper-${ZOOKEEPER_VERSION}-bin.tar.gz"; \
+ download_bin "hadoop.tar.gz" "$HADOOP_HASH" "hadoop/core/hadoop-${HADOOP_VERSION}/hadoop-$HADOOP_VERSION.tar.gz"; \
+ download_bin "accumulo.tar.gz" "${ACCUMULO_HASH}" "accumulo/${ACCUMULO_VERSION}/accumulo-${ACCUMULO_VERSION}-bin.tar.gz";
+
+# Install the dependencies into /opt/
+RUN tar xzf /tmp/hadoop.tar.gz -C /opt/ && ln -s /opt/hadoop-${HADOOP_VERSION} /opt/hadoop
+RUN tar xzf /tmp/apache-zookeeper.tar.gz -C /opt/ && ln -s /opt/apache-zookeeper-${ZOOKEEPER_VERSION}-bin /opt/apache-zookeeper
+RUN tar xzf /tmp/accumulo.tar.gz -C /opt/ && ln -s /opt/accumulo-${ACCUMULO_VERSION} /opt/accumulo && sed -i 's/\${ZOOKEEPER_HOME}\/\*/\${ZOOKEEPER_HOME}\/\*\:\${ZOOKEEPER_HOME}\/lib\/\*/g' /opt/accumulo/conf/accumulo-env.sh
+
+ENV HADOOP_HOME /opt/hadoop
+ENV ZOOKEEPER_HOME /opt/apache-zookeeper
+ENV ACCUMULO_HOME /opt/accumulo
+
+# Add the proxy binary
+COPY target/accumulo-proxy-${ACCUMULO_PROXY_VERSION}-bin.tar.gz /tmp/
Review comment:
Good spot, should be better in the next commit.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] keith-turner commented on a change in pull request #20: WIP: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
keith-turner commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r416662601
##########
File path: DOCKER.md
##########
@@ -0,0 +1,53 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+
+A temporary guide on how to run this up in Docker.
+
+## Build the image using
+Invoke the docker build command to create a container image.
+```commandline
+docker build -t accumulo-proxy:latest .
+```
+
+## Default Configuration
+By default, the container image expects the following to be true:
+1. Your accumulo instance is named "myinstance"
+2. Your zookeeper is available (and reachable from the container) at localhost:2181
+
+## Custom proxy.properties
+If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is:
+```commandline
+docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --add-host"FQDN:IP" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Accumulo Instance Configuration
+In order for Thrift to communicate with the Accumulo instance you will need to provide the container with the FQDN and IP of the Accumulo instance and its relevant servers (e.g. tservers) to allow it to resolve the required DNS.
+
+If you are running an Accumulo instance with more than one tserver you should add each tserver's entry with a new `--add-host "FQDN:IP"` entry.
Review comment:
Could have a sentence like the following.
```
Instead of having an `--add-host` option for each Accumulo server, a single `--network="host"` may work in some environments.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] keith-turner commented on a change in pull request #20: WIP: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
keith-turner commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417370723
##########
File path: DOCKER.md
##########
@@ -0,0 +1,110 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+
+A temporary guide on how to run this up in Docker.
+
+## Build the image using
+Invoke the docker build command to create a container image.
+```commandline
+docker build -t accumulo-proxy:latest .
+```
+
+## Default Configuration and Quickstart
+By default, the container image expects the following to be true:
+1. Your Accumulo instance name is "myinstance"
+2. Your zookeeper is available (and reachable from the container) at localhost:2181
+
+You can start the proxy using:
+```commandline
+docker run --rm -d -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest;
+```
+
+## Custom proxy.properties
+If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is:
+```commandline
+docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --network="host" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Networking configuration
+Container networking can be a very specialised subject therefore we document two common practices that should cover the majority of use cases for development.
+
+The proxy container must be able to access both Accumulo and Zookeeper.
+
+The Zookeeper location can be configured in the `conf/proxy.properties` file, so you can override this to an acceptable value (see "Custom proxy.properties" section)
+
+In order to communicate with Accumulo the container will need to be able to resolve the FQDN that the servers have registered in Zookeeper. If using [fluo-uno](https://github.com/apache/fluo-uno) this is very likely the hostname of your development environment. We'll call this my.host.com and IP 192.168.0.1 for the rest of this document.
Review comment:
This is a really nice high level overview of the networking in Accumulo, from the perspective of an Accumulo client. This could make a good Accumulo blog post that these docs could reference. Could add a bit to the blog post about looking in ZK and scanning the metadata table to see what address servers registered with.
##########
File path: DOCKER.md
##########
@@ -0,0 +1,53 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+
+A temporary guide on how to run this up in Docker.
+
+## Build the image using
+Invoke the docker build command to create a container image.
+```commandline
+docker build -t accumulo-proxy:latest .
+```
+
+## Default Configuration
+By default, the container image expects the following to be true:
+1. Your accumulo instance is named "myinstance"
+2. Your zookeeper is available (and reachable from the container) at localhost:2181
+
+## Custom proxy.properties
+If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is:
+```commandline
+docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --add-host"FQDN:IP" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Accumulo Instance Configuration
+In order for Thrift to communicate with the Accumulo instance you will need to provide the container with the FQDN and IP of the Accumulo instance and its relevant servers (e.g. tservers) to allow it to resolve the required DNS.
+
+If you are running an Accumulo instance with more than one tserver you should add each tserver's entry with a new `--add-host "FQDN:IP"` entry.
Review comment:
> I decided to re-write the whole section covering networking to try and give a bit more clarity around options but I also tried to avoid re-writing the docker documentation, hopefully this strikes a balance.
The documentation is really nice. IMO a good goal for this is a minimum amount of docker documentation to help someone get started quickly. I think what you wrote strikes a nice balance.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] volmasoft commented on a change in pull request #20: WIP: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
volmasoft commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r416597585
##########
File path: DOCKER.md
##########
@@ -0,0 +1,53 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+
+A temporary guide on how to run this up in Docker.
+
+## Build the image using
+Invoke the docker build command to create a container image.
+```commandline
+docker build -t accumulo-proxy:latest .
+```
+
+## Default Configuration
+By default, the container image expects the following to be true:
+1. Your accumulo instance is named "myinstance"
+2. Your zookeeper is available (and reachable from the container) at localhost:2181
+
+## Custom proxy.properties
+If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is:
+```commandline
+docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --add-host"FQDN:IP" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Accumulo Instance Configuration
+In order for Thrift to communicate with the Accumulo instance you will need to provide the container with the FQDN and IP of the Accumulo instance and its relevant servers (e.g. tservers) to allow it to resolve the required DNS.
+
+If you are running an Accumulo instance with more than one tserver you should add each tserver's entry with a new `--add-host "FQDN:IP"` entry.
Review comment:
@keith-turner so that's true in certain environments, this is where I think we have to make a decision about what to support/not support.
In standard linux generally the docker configuration is fairly close to bare metal.
If you use Windows or in my case Mac generally you run Docker 'technically' inside a VM, therefore when you attach the host network it's still not technically the 'host (physical machine)' network.
So whilst adding `--network="host"` into the instructions it'll only actually work for a subset of environments, adding the `--add-host` would work for a wider number and also allows for some isolation of resources (maybe a step too far for dev).
Also this all changes again if we look towards any other orchestration tooling such as docker-compose or Kubernetes as both of those have different working configurations which then need different tuning to work.
---
Given this is starting to creep into environment specific, I might comment on both options in the DOCKER.md and let the caller decide how they want to start their environment, but I'll try and give some hints as to what people may wish to consider.
For now, I don't think many of us are intending Docker to be a simple non-technically orientated mechanism to setup so we can probably get away with giving options for now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] volmasoft commented on a change in pull request #20: WIP: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
volmasoft commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417415282
##########
File path: DOCKER.md
##########
@@ -0,0 +1,110 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+
+A temporary guide on how to run this up in Docker.
+
+## Build the image using
+Invoke the docker build command to create a container image.
+```commandline
+docker build -t accumulo-proxy:latest .
+```
+
+## Default Configuration and Quickstart
+By default, the container image expects the following to be true:
+1. Your Accumulo instance name is "myinstance"
+2. Your zookeeper is available (and reachable from the container) at localhost:2181
+
+You can start the proxy using:
+```commandline
+docker run --rm -d -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest;
+```
+
+## Custom proxy.properties
+If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is:
+```commandline
+docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --network="host" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Networking configuration
+Container networking can be a very specialised subject therefore we document two common practices that should cover the majority of use cases for development.
+
+The proxy container must be able to access both Accumulo and Zookeeper.
+
+The Zookeeper location can be configured in the `conf/proxy.properties` file, so you can override this to an acceptable value (see "Custom proxy.properties" section)
+
+In order to communicate with Accumulo the container will need to be able to resolve the FQDN that the servers have registered in Zookeeper. If using [fluo-uno](https://github.com/apache/fluo-uno) this is very likely the hostname of your development environment. We'll call this my.host.com and IP 192.168.0.1 for the rest of this document.
Review comment:
I agree, if someone fancies writing it I'm more than happy to link it 👍
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] mjwall commented on a change in pull request #20: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
mjwall commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417551792
##########
File path: Dockerfile
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM openjdk:8
+
+EXPOSE 42424
+
+WORKDIR /opt/accumulo-proxy
+
+ARG HADOOP_VERSION=3.2.1
+ARG ZOOKEEPER_VERSION=3.5.7
+ARG ACCUMULO_VERSION=2.0.0
+ARG ACCUMULO_PROXY_VERSION=2.0.0-SNAPSHOT
+
+ARG HADOOP_HASH=a57962a24d178193349917730bf95cdc99bde9df
+ARG ZOOKEEPER_HASH=619928c8553b62775119e3d7d143a4714a160365
+ARG ACCUMULO_HASH=b72bf5c3dcaa25387933a032925046234f30e17a
+
+# Download from Apache mirrors instead of archive #9
+ENV APACHE_DIST_URLS \
+ https://www.apache.org/dyn/closer.cgi?action=download&filename= \
+# if the version is outdated (or we're grabbing the .asc file), we might have to pull from the dist/archive :/
+ https://www-us.apache.org/dist/ \
+ https://www.apache.org/dist/ \
+ https://archive.apache.org/dist/
+
+RUN set -eux; \
+ download_bin() { \
+ local f="$1"; shift; \
+ local hash="$1"; shift; \
+ local distFile="$1"; shift; \
+ local success=; \
+ local distUrl=; \
+ for distUrl in ${APACHE_DIST_URLS}; do \
+ if wget -nv -O "/tmp/${f}" "${distUrl}${distFile}"; then \
+ success=1; \
+ # Checksum the download
+ echo "${hash}" "/tmp/${f}" | sha1sum -c -; \
+ break; \
+ fi; \
+ done; \
+ [ -n "${success}" ]; \
+ };\
+ \
+ download_bin "apache-zookeeper.tar.gz" "${ZOOKEEPER_HASH}" "zookeeper/zookeeper-${ZOOKEEPER_VERSION}/apache-zookeeper-${ZOOKEEPER_VERSION}-bin.tar.gz"; \
+ download_bin "hadoop.tar.gz" "$HADOOP_HASH" "hadoop/core/hadoop-${HADOOP_VERSION}/hadoop-$HADOOP_VERSION.tar.gz"; \
+ download_bin "accumulo.tar.gz" "${ACCUMULO_HASH}" "accumulo/${ACCUMULO_VERSION}/accumulo-${ACCUMULO_VERSION}-bin.tar.gz";
+
+# Install the dependencies into /opt/
Review comment:
:+1:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] keith-turner commented on pull request #20: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
keith-turner commented on pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#issuecomment-622427102
Thanks for the contribution @volmasoft. If you would like to be added as a contributor on the [people page](https://accumulo.apache.org/people/), please [submit a PR to add yourself](https://github.com/apache/accumulo-website/edit/master/pages/people.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] keith-turner commented on a change in pull request #20: WIP: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
keith-turner commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417431387
##########
File path: DOCKER.md
##########
@@ -0,0 +1,110 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+
+A temporary guide on how to run this up in Docker.
+
+## Build the image using
+Invoke the docker build command to create a container image.
+```commandline
+docker build -t accumulo-proxy:latest .
+```
+
+## Default Configuration and Quickstart
+By default, the container image expects the following to be true:
+1. Your Accumulo instance name is "myinstance"
+2. Your zookeeper is available (and reachable from the container) at localhost:2181
+
+You can start the proxy using:
+```commandline
+docker run --rm -d -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest;
+```
+
+## Custom proxy.properties
+If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is:
+```commandline
+docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --network="host" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Networking configuration
+Container networking can be a very specialised subject therefore we document two common practices that should cover the majority of use cases for development.
+
+The proxy container must be able to access both Accumulo and Zookeeper.
+
+The Zookeeper location can be configured in the `conf/proxy.properties` file, so you can override this to an acceptable value (see "Custom proxy.properties" section)
+
+In order to communicate with Accumulo the container will need to be able to resolve the FQDN that the servers have registered in Zookeeper. If using [fluo-uno](https://github.com/apache/fluo-uno) this is very likely the hostname of your development environment. We'll call this my.host.com and IP 192.168.0.1 for the rest of this document.
Review comment:
I may give it a try.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] volmasoft commented on a change in pull request #20: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
volmasoft commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417548700
##########
File path: Dockerfile
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM openjdk:8
+
+EXPOSE 42424
+
+WORKDIR /opt/accumulo-proxy
+
+ARG HADOOP_VERSION=3.2.1
+ARG ZOOKEEPER_VERSION=3.5.7
+ARG ACCUMULO_VERSION=2.0.0
+ARG ACCUMULO_PROXY_VERSION=2.0.0-SNAPSHOT
+
+ARG HADOOP_HASH=a57962a24d178193349917730bf95cdc99bde9df
+ARG ZOOKEEPER_HASH=619928c8553b62775119e3d7d143a4714a160365
+ARG ACCUMULO_HASH=b72bf5c3dcaa25387933a032925046234f30e17a
+
+# Download from Apache mirrors instead of archive #9
+ENV APACHE_DIST_URLS \
+ https://www.apache.org/dyn/closer.cgi?action=download&filename= \
+# if the version is outdated (or we're grabbing the .asc file), we might have to pull from the dist/archive :/
+ https://www-us.apache.org/dist/ \
+ https://www.apache.org/dist/ \
+ https://archive.apache.org/dist/
+
+RUN set -eux; \
+ download_bin() { \
+ local f="$1"; shift; \
+ local hash="$1"; shift; \
+ local distFile="$1"; shift; \
+ local success=; \
+ local distUrl=; \
+ for distUrl in ${APACHE_DIST_URLS}; do \
+ if wget -nv -O "/tmp/${f}" "${distUrl}${distFile}"; then \
+ success=1; \
+ # Checksum the download
+ echo "${hash}" "/tmp/${f}" | sha1sum -c -; \
+ break; \
+ fi; \
+ done; \
+ [ -n "${success}" ]; \
+ };\
+ \
+ download_bin "apache-zookeeper.tar.gz" "${ZOOKEEPER_HASH}" "zookeeper/zookeeper-${ZOOKEEPER_VERSION}/apache-zookeeper-${ZOOKEEPER_VERSION}-bin.tar.gz"; \
+ download_bin "hadoop.tar.gz" "$HADOOP_HASH" "hadoop/core/hadoop-${HADOOP_VERSION}/hadoop-$HADOOP_VERSION.tar.gz"; \
+ download_bin "accumulo.tar.gz" "${ACCUMULO_HASH}" "accumulo/${ACCUMULO_VERSION}/accumulo-${ACCUMULO_VERSION}-bin.tar.gz";
+
+# Install the dependencies into /opt/
Review comment:
For consistency with [accumulo-docker](https://github.com/apache/accumulo-docker/blob/master/Dockerfile) I'd like to keep them as separate commands for now, noting that the cache layers are going to be bigger.
I certainly agree with you and think I will follow up with another ticket that implements the change in both here and accumulo-proxy.
I'd like to ideally keep consistency within the Accumulo project, we're currently discussing just that in an issue here if you have any views https://github.com/apache/accumulo-proxy/issues/21
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] keith-turner commented on a change in pull request #20: WIP: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
keith-turner commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417484321
##########
File path: DOCKER.md
##########
@@ -0,0 +1,114 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+
+A temporary guide on how to run this up in Docker.
Review comment:
```suggestion
A temporary guide on how to run this in Docker.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] keith-turner commented on a change in pull request #20: WIP: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
keith-turner commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417516086
##########
File path: DOCKER.md
##########
@@ -0,0 +1,114 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+
+A temporary guide on how to run this up in Docker.
Review comment:
This is looking really good. I gave everything a quick once over and saw a few things. Do you still consider this a WIP?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] keith-turner commented on a change in pull request #20: WIP: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
keith-turner commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r416031827
##########
File path: DOCKER.md
##########
@@ -0,0 +1,53 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+
+A temporary guide on how to run this up in Docker.
+
+## Build the image using
+Invoke the docker build command to create a container image.
+```commandline
+docker build -t accumulo-proxy:latest .
+```
+
+## Default Configuration
+By default, the container image expects the following to be true:
+1. Your accumulo instance is named "myinstance"
+2. Your zookeeper is available (and reachable from the container) at localhost:2181
+
+## Custom proxy.properties
+If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is:
+```commandline
+docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --add-host"FQDN:IP" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Accumulo Instance Configuration
+In order for Thrift to communicate with the Accumulo instance you will need to provide the container with the FQDN and IP of the Accumulo instance and its relevant servers (e.g. tservers) to allow it to resolve the required DNS.
+
+If you are running an Accumulo instance with more than one tserver you should add each tserver's entry with a new `--add-host "FQDN:IP"` entry.
Review comment:
Instead of adding an entry for each tserver, may be able to use the the docker run option `--network="host"`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] volmasoft commented on a change in pull request #20: WIP: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
volmasoft commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r416803798
##########
File path: DOCKER.md
##########
@@ -0,0 +1,53 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+
+A temporary guide on how to run this up in Docker.
+
+## Build the image using
+Invoke the docker build command to create a container image.
+```commandline
+docker build -t accumulo-proxy:latest .
+```
+
+## Default Configuration
+By default, the container image expects the following to be true:
+1. Your accumulo instance is named "myinstance"
+2. Your zookeeper is available (and reachable from the container) at localhost:2181
+
+## Custom proxy.properties
+If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is:
+```commandline
+docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --add-host"FQDN:IP" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Accumulo Instance Configuration
+In order for Thrift to communicate with the Accumulo instance you will need to provide the container with the FQDN and IP of the Accumulo instance and its relevant servers (e.g. tservers) to allow it to resolve the required DNS.
+
+If you are running an Accumulo instance with more than one tserver you should add each tserver's entry with a new `--add-host "FQDN:IP"` entry.
Review comment:
Take a look at the latest update, I decided to re-write the whole section covering networking to try and give a bit more clarity around options but I also tried to avoid re-writing the docker documentation, hopefully this strikes a balance.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] volmasoft commented on a change in pull request #20: WIP: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
volmasoft commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r416804624
##########
File path: DOCKER.md
##########
@@ -0,0 +1,53 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+
+A temporary guide on how to run this up in Docker.
+
+## Build the image using
+Invoke the docker build command to create a container image.
+```commandline
+docker build -t accumulo-proxy:latest .
+```
+
+## Default Configuration
+By default, the container image expects the following to be true:
+1. Your accumulo instance is named "myinstance"
+2. Your zookeeper is available (and reachable from the container) at localhost:2181
+
+## Custom proxy.properties
+If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is:
+```commandline
+docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --add-host"FQDN:IP" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Accumulo Instance Configuration
+In order for Thrift to communicate with the Accumulo instance you will need to provide the container with the FQDN and IP of the Accumulo instance and its relevant servers (e.g. tservers) to allow it to resolve the required DNS.
+
+If you are running an Accumulo instance with more than one tserver you should add each tserver's entry with a new `--add-host "FQDN:IP"` entry.
Review comment:
I'm going to raise an issue to capture the proxy.properties overrides from the command line, I would like to discuss potential options before implementing these.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] mjwall commented on a change in pull request #20: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
mjwall commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417524869
##########
File path: Dockerfile
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM openjdk:8
+
+EXPOSE 42424
+
+WORKDIR /opt/accumulo-proxy
+
+ARG HADOOP_VERSION=3.2.1
+ARG ZOOKEEPER_VERSION=3.5.7
+ARG ACCUMULO_VERSION=2.0.0
+ARG ACCUMULO_PROXY_VERSION=2.0.0-SNAPSHOT
+
+ARG HADOOP_HASH=a57962a24d178193349917730bf95cdc99bde9df
+ARG ZOOKEEPER_HASH=619928c8553b62775119e3d7d143a4714a160365
+ARG ACCUMULO_HASH=b72bf5c3dcaa25387933a032925046234f30e17a
+
+# Download from Apache mirrors instead of archive #9
+ENV APACHE_DIST_URLS \
+ https://www.apache.org/dyn/closer.cgi?action=download&filename= \
+# if the version is outdated (or we're grabbing the .asc file), we might have to pull from the dist/archive :/
+ https://www-us.apache.org/dist/ \
+ https://www.apache.org/dist/ \
+ https://archive.apache.org/dist/
+
+RUN set -eux; \
+ download_bin() { \
+ local f="$1"; shift; \
+ local hash="$1"; shift; \
+ local distFile="$1"; shift; \
+ local success=; \
+ local distUrl=; \
+ for distUrl in ${APACHE_DIST_URLS}; do \
+ if wget -nv -O "/tmp/${f}" "${distUrl}${distFile}"; then \
+ success=1; \
+ # Checksum the download
+ echo "${hash}" "/tmp/${f}" | sha1sum -c -; \
+ break; \
+ fi; \
+ done; \
+ [ -n "${success}" ]; \
+ };\
+ \
+ download_bin "apache-zookeeper.tar.gz" "${ZOOKEEPER_HASH}" "zookeeper/zookeeper-${ZOOKEEPER_VERSION}/apache-zookeeper-${ZOOKEEPER_VERSION}-bin.tar.gz"; \
+ download_bin "hadoop.tar.gz" "$HADOOP_HASH" "hadoop/core/hadoop-${HADOOP_VERSION}/hadoop-$HADOOP_VERSION.tar.gz"; \
+ download_bin "accumulo.tar.gz" "${ACCUMULO_HASH}" "accumulo/${ACCUMULO_VERSION}/accumulo-${ACCUMULO_VERSION}-bin.tar.gz";
+
+# Install the dependencies into /opt/
Review comment:
I don't know if it matters a whole lot, but you could combine the tar xzf lines with the above line and then remove the files you downloaded which would make the layer smaller. Maybe the size doesn't matter since you are not pushing anywhere
##########
File path: Dockerfile
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM openjdk:8
+
+EXPOSE 42424
+
+WORKDIR /opt/accumulo-proxy
+
+ARG HADOOP_VERSION=3.2.1
+ARG ZOOKEEPER_VERSION=3.5.7
+ARG ACCUMULO_VERSION=2.0.0
+ARG ACCUMULO_PROXY_VERSION=2.0.0-SNAPSHOT
+
+ARG HADOOP_HASH=a57962a24d178193349917730bf95cdc99bde9df
+ARG ZOOKEEPER_HASH=619928c8553b62775119e3d7d143a4714a160365
+ARG ACCUMULO_HASH=b72bf5c3dcaa25387933a032925046234f30e17a
+
+# Download from Apache mirrors instead of archive #9
+ENV APACHE_DIST_URLS \
+ https://www.apache.org/dyn/closer.cgi?action=download&filename= \
+# if the version is outdated (or we're grabbing the .asc file), we might have to pull from the dist/archive :/
+ https://www-us.apache.org/dist/ \
+ https://www.apache.org/dist/ \
+ https://archive.apache.org/dist/
+
+RUN set -eux; \
+ download_bin() { \
+ local f="$1"; shift; \
+ local hash="$1"; shift; \
+ local distFile="$1"; shift; \
+ local success=; \
+ local distUrl=; \
+ for distUrl in ${APACHE_DIST_URLS}; do \
+ if wget -nv -O "/tmp/${f}" "${distUrl}${distFile}"; then \
+ success=1; \
+ # Checksum the download
+ echo "${hash}" "/tmp/${f}" | sha1sum -c -; \
+ break; \
+ fi; \
+ done; \
+ [ -n "${success}" ]; \
+ };\
+ \
+ download_bin "apache-zookeeper.tar.gz" "${ZOOKEEPER_HASH}" "zookeeper/zookeeper-${ZOOKEEPER_VERSION}/apache-zookeeper-${ZOOKEEPER_VERSION}-bin.tar.gz"; \
+ download_bin "hadoop.tar.gz" "$HADOOP_HASH" "hadoop/core/hadoop-${HADOOP_VERSION}/hadoop-$HADOOP_VERSION.tar.gz"; \
+ download_bin "accumulo.tar.gz" "${ACCUMULO_HASH}" "accumulo/${ACCUMULO_VERSION}/accumulo-${ACCUMULO_VERSION}-bin.tar.gz";
+
+# Install the dependencies into /opt/
+RUN tar xzf /tmp/hadoop.tar.gz -C /opt/ && ln -s /opt/hadoop-${HADOOP_VERSION} /opt/hadoop
+RUN tar xzf /tmp/apache-zookeeper.tar.gz -C /opt/ && ln -s /opt/apache-zookeeper-${ZOOKEEPER_VERSION}-bin /opt/apache-zookeeper
+RUN tar xzf /tmp/accumulo.tar.gz -C /opt/ && ln -s /opt/accumulo-${ACCUMULO_VERSION} /opt/accumulo && sed -i 's/\${ZOOKEEPER_HOME}\/\*/\${ZOOKEEPER_HOME}\/\*\:\${ZOOKEEPER_HOME}\/lib\/\*/g' /opt/accumulo/conf/accumulo-env.sh
+
+ENV HADOOP_HOME /opt/hadoop
+ENV ZOOKEEPER_HOME /opt/apache-zookeeper
+ENV ACCUMULO_HOME /opt/accumulo
+
+# Add the proxy binary
+COPY target/accumulo-proxy-${ACCUMULO_PROXY_VERSION}-bin.tar.gz /tmp/
Review comment:
You need to run `mvn package` successfully first or this file will not be here. Not sure if there is a way to check and fail with a helpful message. Might want to mention that in the DOCKER.md as well.
##########
File path: DOCKER.md
##########
@@ -0,0 +1,122 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+This documentation covers how to stand up [accumulo-proxy](https://github.com/apache/accumulo-proxy/) within a Docker container.
+
+The guide covers:
+* Building the image
+* Configuring the `proxy.properties` file
+* Selecting an appropriate networking choice
+* Starting and stopping the container
+* Basic troubleshooting tips
+
+It is not recommended using this guide for a production instance of accumulo-proxy at this time.
+
+## Build the image using
+Invoke the docker build command to create a container image.
+```commandline
+docker build -t accumulo-proxy:latest .
+```
+
+## Default Configuration and Quickstart
+By default, the container image expects the following to be true:
+1. Your Accumulo instance name is "myinstance"
+2. Your ZooKeeper is available (and reachable from the container) at localhost:2181
+
+You can start the proxy using:
+```commandline
+docker run --rm -d -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest;
+```
+
+## Custom proxy.properties
+If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is:
+```commandline
+docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --network="host" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Networking configuration
+Container networking can be a very specialised subject therefore we document two common practices that should cover the majority of use cases for development.
+
+The proxy container must be able to access both Accumulo and ZooKeeper.
+
+The ZooKeeper location can be configured in the `conf/proxy.properties` file, so you can override this to an acceptable value (see "Custom proxy.properties" section)
+
+In order to communicate with Accumulo the container will need to be able to resolve the FQDN that the servers have registered in ZooKeeper. If using [fluo-uno](https://github.com/apache/fluo-uno) this is very likely the hostname of your development environment. We'll call this my.host.com and IP 192.168.0.1 for the rest of this document.
+
+### Host networking
+
+Host networking is the simplest mechanism but generally only works for linux hosts where docker has been installed on 'bare metal' e.g. through an RPM.
+
+You can test if this will work for you by executing the following:
+```commandline
+# Start the accumulo-proxy container and enter it
+docker run -it --rm -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest bash;
+
+# Install telnet and verify if you can connect to my.host.com:9995
+apt-get update && apt-get install telnet;
+telnet my.host.com 9995
Review comment:
Maybe mention this is one way to check connectivity. Could also use curl, something like `curl telnet://my.host.com:9995`. Perhaps give some sample output of what a correct connection looks like.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] keith-turner commented on a change in pull request #20: WIP: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
keith-turner commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417483071
##########
File path: DOCKER.md
##########
@@ -0,0 +1,114 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+
+A temporary guide on how to run this up in Docker.
+
+## Build the image using
+Invoke the docker build command to create a container image.
+```commandline
+docker build -t accumulo-proxy:latest .
+```
+
+## Default Configuration and Quickstart
+By default, the container image expects the following to be true:
+1. Your Accumulo instance name is "myinstance"
+2. Your zookeeper is available (and reachable from the container) at localhost:2181
+
+You can start the proxy using:
+```commandline
+docker run --rm -d -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest;
+```
+
+## Custom proxy.properties
+If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is:
+```commandline
+docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --network="host" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Networking configuration
+Container networking can be a very specialised subject therefore we document two common practices that should cover the majority of use cases for development.
+
+The proxy container must be able to access both Accumulo and Zookeeper.
+
+The Zookeeper location can be configured in the `conf/proxy.properties` file, so you can override this to an acceptable value (see "Custom proxy.properties" section)
+
+In order to communicate with Accumulo the container will need to be able to resolve the FQDN that the servers have registered in Zookeeper. If using [fluo-uno](https://github.com/apache/fluo-uno) this is very likely the hostname of your development environment. We'll call this my.host.com and IP 192.168.0.1 for the rest of this document.
+
+### Host networking
+
+Host networking is the simplest mechanism but generally only works for linux hosts where docker has been installed on 'bare metal' e.g. through an RPM.
+
+You can test if this will work for you by executing the following:
+```commandline
+# Start the accumulo-proxy container and enter it
+docker run -it --rm -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest bash;
+
+# Install telnet and verify if you can connect to my.host.com:9995
+apt-get update && apt-get install telnet;
+telnet my.host.com 9995
+```
+
+If telnet can connect, then your container can resolve `my.host.com` correctly with host networking and therefore you can append `--network="host"` to your docker commands.
+
+An example of using host networking:
+```commandline
+docker run --rm -d -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest
+```
+
+Note: You do not need to map your ports (-p) if using host networking, but we include it for clarity.
+
+For more details see the official docker documentation: [Use host Networking](https://docs.docker.com/network/host)
+
+### Non-Host networking
+
+If you run outside of a single node linux installation, e.g. Docker for Mac, Docker for Windows or use a VM to isolate your docker engine then you will likely need to take this path.
+
+Docker allows you to supply additional addresses to be resolved by the container, and these are automatically added by Docker to the /etc/hosts
+
+For each host add a `--add-host FQDN:IP` entry to your docker run command, you can add multiple entries if need to, see the official docs covering [network settings](https://docs.docker.com/engine/reference/run/#network-settings).
+
+An example of using this approach:
+
+```commandline
+docker run --rm -d -p 42424:42424 --add-host "my.host.com:192.168.0.1" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Cleanup
+Once completed you should stop and remove the container.
+```commandline
+docker stop accumulo-proxy;
+docker rm accumulo-proxy;
+```
+
+## Troubleshooting
+It can often be difficult to know where to start with troubleshooting inside containers, if you need to enter the container without starting the proxy we support this:
+```commandline
+docker run -it --rm -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest bash
+```
+
+The container is very slim so if need be you can add additional tools using `apt`.
+
+If you wish to manually execute the accumulo-proxy in you can:
Review comment:
```suggestion
If you wish to manually execute the accumulo-proxy in the container you can:
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] volmasoft commented on a change in pull request #20: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
volmasoft commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417517262
##########
File path: DOCKER.md
##########
@@ -0,0 +1,114 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+
+A temporary guide on how to run this up in Docker.
Review comment:
I think I'm happy for it to ship, so any last minute things let me know and I'll aim to fix them today/tomorrow, otherwise I'm happy for it to merge whenever.
I removed WIP from the title of the pull request now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-proxy] volmasoft commented on a change in pull request #20: accumulo-proxy docker container
Posted by GitBox <gi...@apache.org>.
volmasoft commented on a change in pull request #20:
URL: https://github.com/apache/accumulo-proxy/pull/20#discussion_r417546058
##########
File path: DOCKER.md
##########
@@ -0,0 +1,122 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# accumulo-proxy-docker
+This documentation covers how to stand up [accumulo-proxy](https://github.com/apache/accumulo-proxy/) within a Docker container.
+
+The guide covers:
+* Building the image
+* Configuring the `proxy.properties` file
+* Selecting an appropriate networking choice
+* Starting and stopping the container
+* Basic troubleshooting tips
+
+It is not recommended using this guide for a production instance of accumulo-proxy at this time.
+
+## Build the image using
+Invoke the docker build command to create a container image.
+```commandline
+docker build -t accumulo-proxy:latest .
+```
+
+## Default Configuration and Quickstart
+By default, the container image expects the following to be true:
+1. Your Accumulo instance name is "myinstance"
+2. Your ZooKeeper is available (and reachable from the container) at localhost:2181
+
+You can start the proxy using:
+```commandline
+docker run --rm -d -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest;
+```
+
+## Custom proxy.properties
+If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is:
+```commandline
+docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --network="host" --name accumulo-proxy accumulo-proxy:latest
+```
+
+## Networking configuration
+Container networking can be a very specialised subject therefore we document two common practices that should cover the majority of use cases for development.
+
+The proxy container must be able to access both Accumulo and ZooKeeper.
+
+The ZooKeeper location can be configured in the `conf/proxy.properties` file, so you can override this to an acceptable value (see "Custom proxy.properties" section)
+
+In order to communicate with Accumulo the container will need to be able to resolve the FQDN that the servers have registered in ZooKeeper. If using [fluo-uno](https://github.com/apache/fluo-uno) this is very likely the hostname of your development environment. We'll call this my.host.com and IP 192.168.0.1 for the rest of this document.
+
+### Host networking
+
+Host networking is the simplest mechanism but generally only works for linux hosts where docker has been installed on 'bare metal' e.g. through an RPM.
+
+You can test if this will work for you by executing the following:
+```commandline
+# Start the accumulo-proxy container and enter it
+docker run -it --rm -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest bash;
+
+# Install telnet and verify if you can connect to my.host.com:9995
+apt-get update && apt-get install telnet;
+telnet my.host.com 9995
Review comment:
The old adage of "you learn something new every day" still rings true.
I've updated the latest commit to use your approach, it means there's less faffing about with installing telnet 👍
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org