You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by gr...@apache.org on 2019/02/04 16:14:10 UTC
[kudu] branch master updated (2551c77 -> ce653d4)
This is an automated email from the ASF dual-hosted git repository.
granthenke pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git.
from 2551c77 KUDU-2411: Fix macOS relocation logic
new 725ad7e [docker] Add a kudu user to the kudu image
new ce653d4 [docker] Add a docker-compose example
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docker/Dockerfile | 10 +++++++++
docker/README.adoc | 47 +++++++++++++++++++++++++++++++++++----
docker/docker-compose.yml | 56 +++++++++++++++++++++++++++++++++++++++++++++++
docker/kudu-entrypoint.sh | 9 +++++---
4 files changed, 115 insertions(+), 7 deletions(-)
create mode 100644 docker/docker-compose.yml
[kudu] 01/02: [docker] Add a kudu user to the kudu image
Posted by gr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
granthenke pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 725ad7e981e2f59e7224cf0c81853daa9fae4171
Author: Grant Henke <gr...@apache.org>
AuthorDate: Sat Feb 2 16:37:30 2019 -0600
[docker] Add a kudu user to the kudu image
As a best practice application images/containers should
not run as root if they don’t need to. This patch creates a
kudu user and group and creates/owns the kudu specific
directories.
I also added a tip to the readme about removing old tags.
I found myself needing to do this while working on this
patch.
Change-Id: Ia0d51cf8c6a585f98641de9b91d51e41dec9dbd0
Reviewed-on: http://gerrit.cloudera.org:8080/12344
Tested-by: Kudu Jenkins
Reviewed-by: Adar Dembo <ad...@cloudera.com>
---
docker/Dockerfile | 10 ++++++++++
docker/README.adoc | 7 +++++++
2 files changed, 17 insertions(+)
diff --git a/docker/Dockerfile b/docker/Dockerfile
index 4415908..09583e3 100644
--- a/docker/Dockerfile
+++ b/docker/Dockerfile
@@ -182,7 +182,10 @@ LABEL name="Apache Kudu Build" \
#
FROM base AS kudu
+ARG UID=1000
+ARG GID=1000
ARG INSTALL_DIR="/opt/kudu"
+ARG DATA_DIR="/var/lib/kudu"
# Copy the binaries.
WORKDIR $INSTALL_DIR/bin
@@ -216,6 +219,13 @@ COPY --from=build /kudu/build/latest/lib ./lib
COPY --from=build /kudu/www ./www
COPY ./docker/kudu-entrypoint.sh /
+# Setup the kudu user and create the neccessary directories.
+RUN groupadd -g ${GID} kudu || groupmod -n kudu $(getent group ${GID} | cut -d: -f1) \
+ && useradd --shell /bin/bash -u ${UID} -g ${GID} -m kudu \
+ && chown -R kudu:kudu ${INSTALL_DIR} \
+ && mkdir -p ${DATA_DIR} && chown -R kudu:kudu ${DATA_DIR}
+USER kudu
+
# Add the entrypoint.
ENTRYPOINT ["/kudu-entrypoint.sh"]
CMD ["help"]
diff --git a/docker/README.adoc b/docker/README.adoc
index 27a3847..f762705 100644
--- a/docker/README.adoc
+++ b/docker/README.adoc
@@ -114,6 +114,13 @@ Remove specific images:
$ docker rmi [IMAGE...]
----
+Remove images by tag pattern:
+[source,bash]
+----
+$ TAG_PATTERN="apache/kudu:*"
+$ docker rmi $(docker images -q "$TAG_PATTERN" --format "{{.Repository}}:{{.Tag}}")
+----
+
=== Using the cache from pre-built images
You can tell docker to considered remote or local images in your build
as cache sources. This can be especially useful when the base or
[kudu] 02/02: [docker] Add a docker-compose example
Posted by gr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
granthenke pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit ce653d465bc0f89a212de25280788d420d20f79d
Author: Grant Henke <gr...@apache.org>
AuthorDate: Sat Feb 2 14:11:39 2019 -0600
[docker] Add a docker-compose example
Adds a brief example of how to start, scale, and
stop a Kudu cluster using docker-compose.
Change-Id: Ie3ac3f9fca16b3bddf45ba24f6e6aa1efcee857c
Reviewed-on: http://gerrit.cloudera.org:8080/12343
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Tested-by: Grant Henke <gr...@apache.org>
Tested-by: Kudu Jenkins
---
docker/README.adoc | 40 +++++++++++++++++++++++++++++----
docker/docker-compose.yml | 56 +++++++++++++++++++++++++++++++++++++++++++++++
docker/kudu-entrypoint.sh | 9 +++++---
3 files changed, 98 insertions(+), 7 deletions(-)
diff --git a/docker/README.adoc b/docker/README.adoc
index f762705..2a2a6fc 100644
--- a/docker/README.adoc
+++ b/docker/README.adoc
@@ -41,6 +41,38 @@ Run an image with a bash prompt and remove it on exit:
$ docker run -it --rm apache/kudu:latest /bin/bash
----
+== Running a Kudu cluster with docker-compose
+
+Below is a brief example of using Kudu with docker compose. For more advanced
+guidance please see the docker-compose documentation
+https://docs.docker.com/compose/[here].
+
+Start a Kudu cluster with the following command:
+(This cluster has 3 masters and 3 tablet servers)
+[source,bash]
+----
+docker-compose -f docker/docker-compose.yml up --scale kudu-tserver=3 -d
+----
+
+Find the web UI url for kudu-master-1:
+[source,bash]
+----
+docker ps -q --filter=name=.*kudu-master-1.* | xargs -L1 docker port \
+ | grep "^8051" | cut -d":" -f2 | sed 's/.*/http\:\/\/localhost\:&/'
+----
+
+You can adjust the number of tablet servers up or down by adjusting the scale:
+[source,bash]
+----
+docker-compose -f docker/docker-compose.yml up --scale kudu-tserver=<scale> -d
+----
+
+Shutdown the cluster:
+[source,bash]
+----
+docker-compose -f docker/docker-compose.yml down
+----
+
== Copying container files to the host
It could be useful to copy files from a pre-built container to your host.
@@ -58,15 +90,15 @@ $ docker cp $SOURCE:/kudu/thirdparty /local/kudu/thirdparty
A runtime image with the Kudu binaries and clients pre-installed
and an entrypoint script that enables easily starting Kudu
masters and tservers along with executing other commands.
-Copies the built artifacts and files from the kudu-build image.
+Copies the built artifacts and files from the kudu:build image.
=== apache/kudu:build-[OS]-[VERSION]
An image that has the Kudu source code pre-built.
-Uses the kudu-thirdparty image as a base.
+Uses the kudu:thirdparty image as a base.
=== apache/kudu:thirdparty-[OS]-[VERSION]
An image that has Kudu's thirdparty dependencies built.
-Uses the kudu-base image as a base.
+Uses the kudu:base image as a base.
=== apache/kudu:base-[OS]-[VERSION]
A base image that has all prerequisite libraries for development and runtime
@@ -102,7 +134,7 @@ Remove all dangling images:
$ docker image prune
----
-Remove all Kudu images :
+Remove all Kudu images:
[source,bash]
----
$ docker rmi -f $(docker images -q apache/kudu)
diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml
new file mode 100644
index 0000000..dcf680e
--- /dev/null
+++ b/docker/docker-compose.yml
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+version: "3"
+services:
+ kudu-master-1:
+ image: apache/kudu:latest
+ ports:
+ - "7051"
+ - "8051"
+ command: ["master"]
+ environment:
+ - KUDU_MASTERS=kudu-master-1,kudu-master-2,kudu-master-3
+ kudu-master-2:
+ image: apache/kudu:latest
+ ports:
+ - "7051"
+ - "8051"
+ command: ["master"]
+ environment:
+ - KUDU_MASTERS=kudu-master-1,kudu-master-2,kudu-master-3
+ kudu-master-3:
+ image: apache/kudu:latest
+ ports:
+ - "7051"
+ - "8051"
+ command: ["master"]
+ environment:
+ - KUDU_MASTERS=kudu-master-1,kudu-master-2,kudu-master-3
+ kudu-tserver:
+ image: apache/kudu:latest
+ depends_on:
+ - kudu-master-1
+ - kudu-master-2
+ - kudu-master-3
+ ports:
+ - "7050"
+ - "8050"
+ command: ["tserver"]
+ environment:
+ - KUDU_MASTERS=kudu-master-1,kudu-master-2,kudu-master-3
+ deploy:
+ replicas: 3
\ No newline at end of file
diff --git a/docker/kudu-entrypoint.sh b/docker/kudu-entrypoint.sh
index c2a47a3..e330180 100755
--- a/docker/kudu-entrypoint.sh
+++ b/docker/kudu-entrypoint.sh
@@ -40,12 +40,13 @@ function print_help {
echo " Defines the root data directory to use."
echo " Defaults to /var/lib/kudu."
echo "MASTER_ARGS:"
- echo " Defines the arguments passed to kudu-master. Defaults to DEFAULT_ARGS"
+ echo " Defines the arguments passed to kudu-master."
+ echo " Defaults to the value of the DEFAULT_ARGS environment variable."
echo "TSERVER_ARGS:"
- echo " Defines the arguments passed to kudu-tserver. Defaults to DEFAULT_ARGS"
+ echo " Defines the arguments passed to kudu-tserver."
+ echo " Defaults to the value of the DEFAULT_ARGS environment variable."
echo "DEFAULT_ARGS:"
echo " Defines a recommended base set of arguments."
- exit 0
}
# Define the defaults environment variables.
@@ -85,6 +86,7 @@ function wait_for_master_hosts() {
# If no arguments are passed, print the help.
if [[ -z "$1" ]]; then
print_help
+ exit 1
fi
# Note: we use "master" and "tserver" here so the kudu-master and kudu-tserver
@@ -107,6 +109,7 @@ elif [[ "$1" == "tserver" ]]; then
exec kudu-tserver ${TSERVER_ARGS}
elif [[ "$1" == "help" ]]; then
print_help
+ exit 0
fi
# Support calling anything else in the container.