You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by gr...@apache.org on 2019/01/25 23:16:10 UTC
[kudu] 02/03: [docker] Add an initial docker build
This is an automated email from the ASF dual-hosted git repository.
granthenke pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit bf41627f6f2f4993a99bc7181b1d2d74d70ee90d
Author: Grant Henke <gr...@apache.org>
AuthorDate: Mon Jan 21 19:58:30 2019 -0600
[docker] Add an initial docker build
This patch adds an initial Docker file which defines a
multistage build that supports creating various Docker
images for Apache Kudu development.
This work is experimental and the choices and structure
may change with follow on commits.
Some things these images could be used for include:
- demos/examples
- pre-built binaries including thirdparty
- multi-os testing
- multi-os tools like build_mini_cluster_binaries.sh
- external integration testing
Some of the open tasks for the future are:
- Handle all build types.
- Optimize image sizes.
- Add kudu user to runtime images.
- Add an upload script with good tagging.
- Add healthchecks.
Change-Id: I95497b39e47f07301be75cbadd814656c7e2ea42
Reviewed-on: http://gerrit.cloudera.org:8080/12173
Tested-by: Kudu Jenkins
Reviewed-by: Attila Bukor <ab...@apache.org>
Tested-by: Attila Bukor <ab...@apache.org>
Reviewed-by: Alexey Serbin <as...@cloudera.com>
---
.dockerignore | 91 ++++++++++++
docker/Dockerfile | 315 ++++++++++++++++++++++++++++++++++++++++
docker/Dockerfile-files | 31 ++++
docker/README.adoc | 80 ++++++++++
docker/bootstrap-env.sh | 167 +++++++++++++++++++++
docker/docker-build.sh | 56 +++++++
docker/kudu-entrypoint.sh | 76 ++++++++++
thirdparty/build-definitions.sh | 1 +
8 files changed, 817 insertions(+)
diff --git a/.dockerignore b/.dockerignore
new file mode 100644
index 0000000..423e394
--- /dev/null
+++ b/.dockerignore
@@ -0,0 +1,91 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Ignore everything.
+# It is easier to ensure smaller docker images and avoid cache invalidations
+# to ignore everything and add back the files we want.
+*
+
+# General top level source files.
+!version.txt
+
+# Docker files.
+!docker/bootstrap-env.sh
+!docker/kudu-entrypoint.sh
+
+# Docs files.
+!docs
+# Ignore docs files we don't need in docker.
+docs/design-docs
+docs/whitepaper
+
+# Example files.
+!examples
+# Ensure these directories are completely ignored.
+examples/**/.idea
+examples/**/target
+
+# Java source files.
+!java
+# Ensure these directories are completely ignored.
+java/**/build
+java/**/.gradle
+java/**/.idea
+# Ensure compiled source is ignored.
+java/**/*.jar
+
+# Python source files.
+!python
+# Ensure these directories are completely ignored.
+python/.eggs
+python/.idea
+python/build
+python/dist
+python/sdist
+python/env/
+python/venv/
+# Ensure generated source is ignored.
+python/**/*.c
+python/**/*.cpp
+python/**/*.cmake
+python/kudu/version.py
+python/kudu/config.pxi
+# Ensure compiled source is ignored.
+**/*.py[ocd]
+
+# C++ source files.
+!build-support
+!cmake_modules
+!src
+!CMakeLists.txt
+# Ensure compiled source is ignored.
+**/*.so
+# Ignore build files we don't need in docker.
+build-support/jenkins
+
+# Thirdparty source files.
+!thirdparty/patches
+!thirdparty/scripts
+!thirdparty/*.py
+!thirdparty/*.sh
+!thirdparty/*.txt
+
+# www source files.
+# Don't ignore all the www directory files except those generated by the build.
+!www
+# Tracing files are generated by the thirdparty trace-viewer build.
+www/tracing.*
\ No newline at end of file
diff --git a/docker/Dockerfile b/docker/Dockerfile
new file mode 100644
index 0000000..6835138
--- /dev/null
+++ b/docker/Dockerfile
@@ -0,0 +1,315 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This docker file defines a multistage build that supports creating
+# various docker images for Apache Kudu development.
+#
+# Note: When editing this file, please follow the best practices laid out here:
+# https://docs.docker.com/develop/develop-images/dockerfile_best-practices
+#
+# Note: This file uses the shared label namespace for common labels. See:
+# http://label-schema.org/rc1/
+
+#
+# ---- Base ----
+# Builds a base image that has all prerequisite libraries for
+# development and runtime pre-installed.
+# TODO: Consider a separate runtime-base and buildtime-base to make
+# runtime images smaller.
+#
+ARG BASE_OS=ubuntu:xenial
+FROM $BASE_OS as kudu-base
+
+COPY ./docker/bootstrap-env.sh /
+RUN ./bootstrap-env.sh && rm bootstrap-env.sh
+
+# Common label arguments.
+# VCS_REF is not specified to improve docker caching.
+ARG DOCKERFILE
+ARG MAINTAINER
+ARG URL
+ARG VCS_TYPE
+ARG VCS_URL
+ARG VERSION
+
+LABEL org.label-schema.name="Apache Kudu Base" \
+ org.label-schema.description="A base image that has all prerequisite \
+ libraries for development and runtime pre-installed." \
+ # Common labels.
+ org.label-schema.dockerfile=$DOCKERFILE \
+ org.label-schema.maintainer=$MAINTAINER \
+ org.label-schema.url=$URL \
+ org.label-schema.vcs-type=$VCS_TYPE \
+ org.label-schema.vcs-url=$VCS_URL \
+ org.label-schema.version=$VERSION
+
+#
+# ---- Thirdparty ----
+# Builds an image that has Kudu's thirdparty dependencies built.
+# This is done in its own stage so that docker can cache it and only
+# run it when thirdparty has changes.
+#
+FROM kudu-base AS kudu-thirdparty
+
+WORKDIR /kudu
+# We only copy the needed files for thirdparty so docker can handle caching.
+COPY ./thirdparty thirdparty
+COPY ./build-support/enable_devtoolset.sh \
+ ./build-support/enable_devtoolset_inner.sh \
+ build-support/
+RUN build-support/enable_devtoolset.sh \
+ thirdparty/build-if-necessary.sh \
+ # Remove the files left behind that we don't need.
+ # Remove all the source files except the hive, hadoop, and sentry sources
+ # which are pre-built and symlinked into the installed/common/opt directory.
+ && find thirdparty/src/* -maxdepth 0 -type d \
+ \( ! -name 'hadoop-*' ! -name 'hive-*' ! -name 'apache-sentry-*' \) \
+ -prune -exec rm -rf {} \; \
+ # Remove all the build files except the llvm build which is symlinked into
+ # the clang-toolchain directory.
+ && find thirdparty/build/* -maxdepth 0 -type d ! -name 'llvm-*' -prune -exec rm -rf {} \;
+
+# Common label arguments.
+ARG DOCKERFILE
+ARG MAINTAINER
+ARG URL
+ARG VCS_REF
+ARG VCS_TYPE
+ARG VCS_URL
+ARG VERSION
+
+LABEL name="Apache Kudu Thirdparty" \
+ description="An image that has Kudu's thirdparty dependencies pre-built." \
+ # Common labels.
+ org.label-schema.dockerfile=$DOCKERFILE \
+ org.label-schema.maintainer=$MAINTAINER \
+ org.label-schema.url=$URL \
+ org.label-schema.vcs-ref=$VCS_REF \
+ org.label-schema.vcs-type=$VCS_TYPE \
+ org.label-schema.vcs-url=$VCS_URL \
+ org.label-schema.version=$VERSION
+
+#
+# ---- Build ----
+# Builds an image that has the Kudu source code pre-built.
+# This is useful for generating a runtime image, but can also be a
+# useful base development image.
+#
+FROM kudu-thirdparty AS kudu-build
+
+# TODO: Support other buildtypes.
+ARG BUILD_TYPE=release
+ARG PARALLEL=4
+
+WORKDIR /kudu
+# Copy the C++ build source.
+# We copy the minimal source to optimize docker cache hits.
+COPY ./build-support build-support
+COPY ./docs/support docs/support
+COPY ./cmake_modules cmake_modules
+COPY ./examples/cpp examples/cpp
+COPY ./java/kudu-hive/ java/kudu-hive/
+COPY ./src src
+COPY ./CMakeLists.txt ./version.txt ./
+# Build the c++ code.
+WORKDIR /kudu/build/$BUILD_TYPE
+# Ensure we don't rebuild thirdparty. Instead let docker handle caching.
+ENV NO_REBUILD_THIRDPARTY=1
+RUN ../../build-support/enable_devtoolset.sh \
+ ../../thirdparty/installed/common/bin/cmake \
+ -DCMAKE_BUILD_TYPE=$BUILD_TYPE \
+ # The release build is massive with tests built.
+ -DNO_TESTS=1 \
+ ../.. \
+ && make -j${PARALLEL} \
+ # Install the client libraries for the python build to use.
+ && make install
+
+# Copy the java build source.
+COPY ./java /kudu/java
+# Build the java code.
+WORKDIR /kudu/java
+RUN ./gradlew jar
+
+# Copy the python build source.
+COPY ./python /kudu/python
+# Build the python code.
+WORKDIR /kudu/python
+RUN pip install -r requirements.txt \
+ && python setup.py sdist
+
+# Copy any remaining source files.
+COPY . /kudu
+
+# Common label arguments.
+ARG DOCKERFILE
+ARG MAINTAINER
+ARG URL
+ARG VCS_REF
+ARG VCS_TYPE
+ARG VCS_URL
+ARG VERSION
+
+LABEL name="Apache Kudu Build" \
+ description="An image that has the Kudu source code pre-built." \
+ org.apache.kudu.build.type=$BUILD_TYPE \
+ # Common labels.
+ org.label-schema.dockerfile=$DOCKERFILE \
+ org.label-schema.maintainer=$MAINTAINER \
+ org.label-schema.url=$URL \
+ org.label-schema.vcs-ref=$VCS_REF \
+ org.label-schema.vcs-type=$VCS_TYPE \
+ org.label-schema.vcs-url=$VCS_URL \
+ org.label-schema.version=$VERSION
+
+#
+# ---- Runtime ----
+# Builds a runtime image with the Kudu binaries and clients pre-installed.
+#
+FROM kudu-base AS kudu-runtime
+
+ARG INSTALL_DIR="/opt/kudu"
+
+# Copy the binaries.
+WORKDIR $INSTALL_DIR/bin
+COPY --from=kudu-build \
+ /kudu/build/latest/bin/kudu \
+ /kudu/build/latest/bin/kudu-master \
+ /kudu/build/latest/bin/kudu-tserver \
+ ./
+# Add to the binaries to the path.
+ENV PATH=$INSTALL_DIR/bin/:$PATH
+
+# Copy the python files and install.
+WORKDIR $INSTALL_DIR/python
+COPY --from=kudu-build /usr/local /usr/local/
+ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib64
+COPY --from=kudu-build /kudu/python/dist/kudu-python-*.tar.gz .
+RUN pip install kudu-python-*.tar.gz
+
+# Copy the Java application jars.
+WORKDIR $INSTALL_DIR/java
+COPY --from=kudu-build \
+ /kudu/java/kudu-backup/build/libs/*.jar \
+ /kudu/java/kudu-client-tools/build/libs/*.jar \
+ /kudu/java/kudu-spark-tools/build/libs/*.jar \
+ ./
+
+WORKDIR $INSTALL_DIR
+# Copy the lib files.
+COPY --from=kudu-build /kudu/build/latest/lib ./lib
+# Copy the web files.
+COPY --from=kudu-build /kudu/www ./www
+# Copy the examples files for convenience.
+COPY --from=kudu-build /kudu/examples ./examples
+
+# Common label arguments.
+ARG DOCKERFILE
+ARG MAINTAINER
+ARG URL
+ARG VCS_REF
+ARG VCS_TYPE
+ARG VCS_URL
+ARG VERSION
+
+LABEL name="Apache Kudu Runtime" \
+ description="A runtime image with the Kudu binaries and clients pre-installed." \
+ org.apache.kudu.build.type=$BUILD_TYPE \
+ # Common labels.
+ org.label-schema.dockerfile=$DOCKERFILE \
+ org.label-schema.maintainer=$MAINTAINER \
+ org.label-schema.url=$URL \
+ org.label-schema.vcs-ref=$VCS_REF \
+ org.label-schema.vcs-type=$VCS_TYPE \
+ org.label-schema.vcs-url=$VCS_URL \
+ org.label-schema.version=$VERSION
+
+#
+# ---- kudu-master ----
+# Builds a runtime image with kudu-master as the ENTRYPOINT.
+#
+FROM kudu-runtime AS kudu-master
+
+# RPC port and web interface port.
+EXPOSE 7051 8051
+
+# TODO: Add healthchecks
+
+COPY ./docker/kudu-entrypoint.sh /
+ENTRYPOINT ["/kudu-entrypoint.sh"]
+
+# Common label arguments.
+ARG BUILD_TYPE
+ARG DOCKERFILE
+ARG MAINTAINER
+ARG URL
+ARG VCS_REF
+ARG VCS_TYPE
+ARG VCS_URL
+ARG VERSION
+
+LABEL name="Apache Kudu Master" \
+ description="A runtime image with kudu-master as the ENTRYPOINT" \
+ org.apache.kudu.build.type=$BUILD_TYPE \
+ # Common labels.
+ org.label-schema.dockerfile=$DOCKERFILE \
+ org.label-schema.maintainer=$MAINTAINER \
+ org.label-schema.url=$URL \
+ org.label-schema.vcs-ref=$VCS_REF \
+ org.label-schema.vcs-type=$VCS_TYPE \
+ org.label-schema.vcs-url=$VCS_URL \
+ org.label-schema.version=$VERSION
+
+CMD ["kudu-master"]
+
+#
+# ---- kudu-tserver ----
+# Builds a runtime image with kudu-tserver as the ENTRYPOINT.
+#
+FROM kudu-runtime AS kudu-tserver
+
+# RPC port and web interface port.
+EXPOSE 7050 8050
+
+# TODO: Add healthchecks
+
+COPY ./docker/kudu-entrypoint.sh /
+ENTRYPOINT ["/kudu-entrypoint.sh"]
+
+# Common label arguments.
+ARG BUILD_TYPE
+ARG DOCKERFILE
+ARG MAINTAINER
+ARG URL
+ARG VCS_REF
+ARG VCS_TYPE
+ARG VCS_URL
+ARG VERSION
+
+LABEL name="Apache Kudu Tserver" \
+ description="A runtime image with kudu-tserver as the ENTRYPOINT" \
+ org.apache.kudu.build.type=$BUILD_TYPE \
+ # Common labels.
+ org.label-schema.dockerfile=$DOCKERFILE \
+ org.label-schema.maintainer=$MAINTAINER \
+ org.label-schema.url=$URL \
+ org.label-schema.vcs-ref=$VCS_REF \
+ org.label-schema.vcs-type=$VCS_TYPE \
+ org.label-schema.vcs-url=$VCS_URL \
+ org.label-schema.version=$VERSION
+
+CMD ["kudu-tserver"]
diff --git a/docker/Dockerfile-files b/docker/Dockerfile-files
new file mode 100644
index 0000000..7e143f3
--- /dev/null
+++ b/docker/Dockerfile-files
@@ -0,0 +1,31 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This docker file is a minimal build useful for debugging
+# what files are being included/excluded via the .dockerignore file.
+
+# Usage
+# docker build -f docker/Dockerfile-files -t kudu-files --no-cache .
+# docker run -i -t --rm=true kudu-files /bin/bash
+# $ ls -als
+# $ du -h
+# $ du -sh
+
+FROM centos
+
+WORKDIR /kudu
+COPY . /kudu
\ No newline at end of file
diff --git a/docker/README.adoc b/docker/README.adoc
new file mode 100644
index 0000000..92d206d
--- /dev/null
+++ b/docker/README.adoc
@@ -0,0 +1,80 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+= Kudu Docker Developer Documentation
+
+NOTE: All of this work is experimental and subject to change or removal.
+
+== Getting Started
+
+- Install docker following the instructions https://www.docker.com/get-started[here]
+
+== Building images
+
+NOTE: These sample commands assume running from the project root directory.
+
+Build all the images.
+[source,bash]
+----
+$ ./docker/docker-build.sh
+----
+
+== Running an image
+
+Run an image with a bash prompt and remove it on exit:
+[source,bash]
+----
+$ docker run -i -t --rm=true kudu-build /bin/bash
+----
+
+== Copying container files to the host
+
+It could be useful to copy files from a pre-built container to your host.
+For example, pre-built thirdparty or kudu binaries.
+
+[source,bash]
+----
+$ SOURCE=`docker create kudu-thirdparty`
+$ docker cp $SOURCE:/kudu/thirdparty /local/kudu/thirdparty
+----
+
+== Images
+
+=== kudu-base
+A base image that has all prerequisite libraries for development and runtime
+pre-installed.
+
+=== kudu-thirdparty
+An image that has Kudu's thirdparty dependencies built.
+Uses the kudu-base image as a base.
+
+=== kudu-build
+An image that has the Kudu source code pre-built.
+Uses the kudu-thirdparty image as a base.
+
+=== kudu-runtime
+A runtime image with the Kudu binaries and clients pre-installed.
+Copies the built artifacts and files from the kudu-build image.
+
+=== kudu-master
+A runtime image with kudu-master as the ENTRYPOINT.
+Uses the kudu-runtime image as a base.
+
+=== kudu-tserver
+A runtime image with kudu-tserver as the ENTRYPOINT.
+Uses the kudu-runtime image as a base.
+
diff --git a/docker/bootstrap-env.sh b/docker/bootstrap-env.sh
new file mode 100755
index 0000000..3ebc6d9
--- /dev/null
+++ b/docker/bootstrap-env.sh
@@ -0,0 +1,167 @@
+#!/bin/bash
+##########################################################
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# This script handles bootstrapping a base OS for
+# the Apache Kudu base docker images.
+#
+# TODO: Consider pre-installing nscd to avoid the issue here:
+# https://kudu.apache.org/docs/troubleshooting.html#slow_dns_nscd
+#
+##########################################################
+
+set -xe
+
+# Install the prerequisite libraries, if they are not installed.
+# CentOS/RHEL
+if [[ -n $(which yum) ]]; then
+ # Update the repo.
+ yum update -y
+
+ # Install core build libraries.
+ yum install -y \
+ autoconf \
+ automake \
+ cyrus-sasl-devel \
+ cyrus-sasl-gssapi \
+ cyrus-sasl-plain \
+ flex \
+ gcc \
+ gcc-c++ \
+ gdb \
+ git \
+ java-1.8.0-openjdk-devel \
+ krb5-server \
+ krb5-workstation \
+ libtool \
+ make \
+ openssl-devel \
+ patch \
+ pkgconfig \
+ redhat-lsb-core \
+ rsync \
+ unzip \
+ vim-common \
+ which \
+ wget
+
+ # Install docs build libraries.
+ yum install -y \
+ doxygen \
+ gem \
+ graphviz \
+ ruby-devel \
+ zlib-devel
+
+ # Install and upgrade pip for python development.
+ yum install -y epel-release
+ yum install -y \
+ python-devel \
+ python-pip
+ pip install --upgrade \
+ cython \
+ pip \
+ setuptools
+
+ # To build on a version older than 7.0, the Red Hat Developer Toolset
+ # must be installed (in order to have access to a C++11 capable compiler).
+ OS_MAJOR_VERSION=$(lsb_release -rs | cut -f1 -d.)
+ if [[ "$OS_MAJOR_VERSION" -lt "7" ]]; then
+ DTLS_RPM=rhscl-devtoolset-3-epel-6-x86_64-1-2.noarch.rpm
+ DTLS_RPM_URL=https://www.softwarecollections.org/repos/rhscl/devtoolset-3/epel-6-x86_64/noarch/${DTLS_RPM}
+ wget ${DTLS_RPM_URL} -O ${DTLS_RPM}
+ yum install -y scl-utils ${DTLS_RPM}
+ yum install -y devtoolset-3-toolchain
+ fi
+
+ # Reduce the image size by cleaning up after the install.
+ yum clean all
+ rm -rf /var/cache/yum /tmp/* /var/tmp/*
+# Ubuntu/Debian
+elif [[ -n $(which apt-get) ]]; then
+ # Ensure the Debian frontend is noninteractive.
+ export DEBIAN_FRONTEND=noninteractive
+
+ # Update the repo.
+ apt-get update -y
+
+ # Add the PPA repository for openjdk-8-jdk on ubuntu:trusty
+ apt-get install -y --no-install-recommends software-properties-common
+ add-apt-repository ppa:openjdk-r/ppa
+ apt-get update -y
+
+ # Install core build libraries.
+ # --no-install-recommends keeps the install smaller
+ apt-get install -y --no-install-recommends \
+ autoconf \
+ automake \
+ curl \
+ flex \
+ g++ \
+ gcc \
+ gdb \
+ git \
+ krb5-admin-server \
+ krb5-kdc \
+ krb5-user \
+ libkrb5-dev \
+ libsasl2-dev \
+ libsasl2-modules \
+ libsasl2-modules-gssapi-mit \
+ libssl-dev \
+ libtool \
+ lsb-release \
+ make \
+ ntp \
+ openjdk-8-jdk \
+ openssl \
+ patch \
+ pkg-config \
+ python \
+ rsync \
+ unzip \
+ vim-common
+
+ # Install docs build libraries.
+ apt-get install -y --no-install-recommends \
+ doxygen \
+ gem \
+ graphviz \
+ ruby-dev \
+ xsltproc \
+ zlib1g-dev
+
+ # Install and upgrade pip for python development.
+ apt-get install -y --no-install-recommends \
+ python-dev \
+ python-pip
+ pip install --upgrade \
+ cython \
+ pip \
+ setuptools
+
+ # Reduce the image size by cleaning up after the install.
+ apt-get clean
+ rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
+
+ unset DEBIAN_FRONTEND
+else
+ echo "Unsupported OS"
+ exit 1
+fi
\ No newline at end of file
diff --git a/docker/docker-build.sh b/docker/docker-build.sh
new file mode 100755
index 0000000..2aed7e0
--- /dev/null
+++ b/docker/docker-build.sh
@@ -0,0 +1,56 @@
+#!/bin/bash -xe
+##########################################################
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# This script handles the coordination of building all of
+# the Apache Kudu docker images.
+#
+##########################################################
+
+ROOT=$(cd $(dirname "$BASH_SOURCE")/.. ; pwd)
+
+# Tested options:
+# centos:6
+# centos:7
+# debian:jessie
+# ubuntu:trusty
+# ubuntu:xenial
+# ubuntu:bionic
+BASE_OS=${BASE_OS:=ubuntu:xenial}
+
+VERSION=`cat $ROOT/version.txt`
+VCS_REF=`git rev-parse --short HEAD`
+
+BUILD_ARGS=(
+ --build-arg BASE_OS="$BASE_OS"
+ --build-arg DOCKERFILE="docker/Dockerfile"
+ --build-arg MAINTAINER="Apache Kudu <de...@kudu.apache.org>"
+ --build-arg URL="https://kudu.apache.org"
+ --build-arg VERSION=$VERSION
+ --build-arg VCS_REF=$VCS_REF
+ --build-arg VCS_TYPE="git"
+ --build-arg VCS_URL="https://gitbox.apache.org/repos/asf/kudu.git"
+)
+
+docker build "${BUILD_ARGS[@]}" -f $ROOT/docker/Dockerfile --target kudu-base -t kudu-base $ROOT
+docker build "${BUILD_ARGS[@]}" -f $ROOT/docker/Dockerfile --target kudu-thirdparty -t kudu-thirdparty $ROOT
+docker build "${BUILD_ARGS[@]}" -f $ROOT/docker/Dockerfile --target kudu-build -t kudu-build $ROOT
+docker build "${BUILD_ARGS[@]}" -f $ROOT/docker/Dockerfile --target kudu-runtime -t kudu-runtime $ROOT
+docker build "${BUILD_ARGS[@]}" -f $ROOT/docker/Dockerfile --target kudu-master -t kudu-master $ROOT
+docker build "${BUILD_ARGS[@]}" -f $ROOT/docker/Dockerfile --target kudu-tserver -t kudu-tserver $ROOT
\ No newline at end of file
diff --git a/docker/kudu-entrypoint.sh b/docker/kudu-entrypoint.sh
new file mode 100755
index 0000000..58d99ec
--- /dev/null
+++ b/docker/kudu-entrypoint.sh
@@ -0,0 +1,76 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This script follows the pattern described in the docker best practices here:
+# https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#entrypoint
+#
+# KUDU_FLAGS can be set by users to do more configuration of the
+# kudu-master and kudu-tserver.
+
+set -xe
+
+# Wait until the master hosts can be resolved.
+#
+# Without this Kudu will fail with "Name or service not known" errors
+# on startup.
+#
+# Gives a maximum of 5 attempts/seconds to each host. On failure
+# falls through without failing to still give Kudu a chance to startup
+# or fail on it's own.
+wait_for_master_hosts() {
+ IFS=","
+ for HOST in $KUDU_MASTERS
+ do
+ MAX_ATTEMPTS=5
+ ATTEMPTS=0
+ until `ping -c1 $HOST &>/dev/null;` || [ $ATTEMPTS -eq $MAX_ATTEMPTS ]; do
+ ATTEMPTS=$((ATTEMPTS + 1))
+ sleep 1;
+ done
+ done
+ unset IFS
+}
+
+if [[ "$1" = 'kudu-master' ]]; then
+ # Create the data directory.
+ mkdir -p /var/lib/kudu/master
+ # TODO: Remove use_hybrid_clock=false when ntpd is setup
+ KUDU_OPTS="--master_addresses=$KUDU_MASTERS
+ --fs_wal_dir=/var/lib/kudu/master \
+ --webserver_doc_root=/opt/kudu/www \
+ --logtostderr \
+ --use_hybrid_clock=false \
+ $KUDU_FLAGS"
+ wait_for_master_hosts
+ exec kudu-master $KUDU_OPTS
+elif [[ "$1" = 'kudu-tserver' ]]; then
+ # Create the data directory.
+ mkdir -p /var/lib/kudu/tserver
+ # TODO: Remove use_hybrid_clock=false when ntpd is setup
+ KUDU_OPTS="--tserver_master_addrs=$KUDU_MASTERS
+ --fs_wal_dir=/var/lib/kudu/tserver \
+ --webserver_doc_root=/opt/kudu/www \
+ --logtostderr \
+ --use_hybrid_clock=false \
+ $KUDU_FLAGS"
+ wait_for_master_hosts
+ exec kudu-tserver $KUDU_OPTS
+fi
+
+# Support calling anything else in the container.
+exec "$@"
\ No newline at end of file
diff --git a/thirdparty/build-definitions.sh b/thirdparty/build-definitions.sh
index d0c47ba..2ae2f59 100644
--- a/thirdparty/build-definitions.sh
+++ b/thirdparty/build-definitions.sh
@@ -743,6 +743,7 @@ build_gcovr() {
build_trace_viewer() {
echo Installing trace-viewer into the www directory
+ mkdir -p $TP_DIR/../www/
cp -a $TRACE_VIEWER_SOURCE/tracing.* $TP_DIR/../www/
}