You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by xi...@apache.org on 2019/06/13 17:09:23 UTC
[incubator-pinot] branch master updated: Adding script to build and
publish docker image (#4200)
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
The following commit(s) were added to refs/heads/master by this push:
new 27ad26c Adding script to build and publish docker image (#4200)
27ad26c is described below
commit 27ad26c43114350d5549496b7c5e069f8f607c53
Author: Xiang Fu <fx...@gmail.com>
AuthorDate: Thu Jun 13 10:09:18 2019 -0700
Adding script to build and publish docker image (#4200)
* Adding script to build and publish docker image (#2)
* Address comments
* Adding license header
* Adding thrift support in build image
* update readme
---
docker/Dockerfile | 62 +++++++++++++++++++++
docker/README.md | 116 ++++++++++++++++++++++++++++++++++++++++
docker/docker-build-and-push.sh | 30 +++++++++++
docker/docker-build.sh | 47 ++++++++++++++++
docker/docker-push.sh | 29 ++++++++++
5 files changed, 284 insertions(+)
diff --git a/docker/Dockerfile b/docker/Dockerfile
new file mode 100644
index 0000000..86bb47c
--- /dev/null
+++ b/docker/Dockerfile
@@ -0,0 +1,62 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+FROM openjdk:8
+
+LABEL MAINTAINER=dev@pinot.apache.org
+
+ARG PINOT_BRANCH=master
+ARG PINOT_GIT_URL="https://github.com/apache/incubator-pinot.git"
+RUN echo "Trying to build Pinot from [ ${PINOT_GIT_URL} ] on branch [ ${PINOT_BRANCH} ]"
+ENV PINOT_HOME=/opt/pinot
+ENV PINOT_BUILD_DIR=/opt/pinot-build
+
+# extra dependency for running launcher
+RUN apt-get update && \
+ apt-get install -y --no-install-recommends vim wget curl git maven automake bison flex g++ libboost-all-dev libevent-dev libssl-dev libtool make pkg-config && \
+ rm -rf /var/lib/apt/lists/* && \
+ wget http://archive.apache.org/dist/thrift/0.12.0/thrift-0.12.0.tar.gz -O /tmp/thrift-0.12.0.tar.gz && \
+ tar xfz /tmp/thrift-0.12.0.tar.gz --directory /tmp && \
+ base_dir=`pwd` && \
+ cd /tmp/thrift-0.12.0 && \
+ ./configure --with-cpp=no --with-c_glib=no --with-java=yes --with-python=no --with-ruby=no --with-erlang=no --with-go=no --with-nodejs=no --with-php=no && \
+ make install && \
+ cd ${base_dir}
+
+RUN git clone ${PINOT_GIT_URL} ${PINOT_BUILD_DIR} && \
+ cd ${PINOT_BUILD_DIR} && \
+ git checkout ${PINOT_BRANCH} && \
+ mvn install package -DskipTests -Pbin-dist -Pbuild-shaded-jar && \
+ mkdir -p ${PINOT_HOME}/configs && \
+ mkdir -p ${PINOT_HOME}/data && \
+ cp -r pinot-distribution/target/apache-pinot-*-bin/apache-pinot-*-bin/* ${PINOT_HOME}/. && \
+ chmod +x ${PINOT_HOME}/bin/*.sh && \
+ mvn dependency:purge-local-repository -DactTransitively=false -DreResolve=false --fail-at-end && \
+ rm -rf ${PINOT_BUILD_DIR}
+
+VOLUME ["${PINOT_HOME}/configs", "${PINOT_HOME}/data"]
+
+# expose ports for controller/broker/server/admin
+EXPOSE 9000 8099 8098 8097 8096
+
+WORKDIR ${PINOT_HOME}
+
+ENTRYPOINT ["./bin/pinot-admin.sh"]
+
+CMD ["run"]
diff --git a/docker/README.md b/docker/README.md
new file mode 100644
index 0000000..d0d23d3
--- /dev/null
+++ b/docker/README.md
@@ -0,0 +1,116 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+# docker-pinot
+This is a docker image of [Apache Pinot](https://github.com/apache/incubator-pinot).
+
+## How to build a docker image
+
+There is a docker build script which will build a given Git repo/branch and tag the image.
+
+Usage:
+
+```SHELL
+./docker-build.sh [Docker Tag] [Git Branch] [Pinot Git URL]
+```
+
+This script will check out Pinot Repo `[Pinot Git URL]` on branch `[Git Branch]` and build the docker image for that.
+
+The docker image is tagged as `[Docker Tag]`.
+
+`Docker Tag`: Name and tag your docker image. Default is `pinot:latest`.
+
+`Git Branch`: The Pinot branch to build. Default is `master`.
+
+`Pinot Git URL`: The Pinot Git Repo to build, users can set it to their own fork. Please note that, the URL is `https://` based, not `git://`. Default is the Apache Repo: `https://github.com/apache/incubator-pinot.git`.
+
+* Example of building and tagging a snapshot on your own fork:
+```SHELL
+./docker-build.sh pinot_fork:snapshot-5.2 snapshot-5.2 https://github.com/your_own_fork/pinot.git
+```
+
+* Example of building a release version:
+```SHELL
+./docker-build.sh pinot:release-0.1.0 release-0.1.0 https://github.com/apache/incubator-pinot.git
+```
+
+## How to publish a docker image
+
+Script `docker-push.sh` publishes a given docker image to your docker registry.
+
+In order to push to your own repo, the image needs to be explicitly tagged with the repo name.
+
+* Example of publishing a image to [winedepot/pinot](https://cloud.docker.com/u/winedepot/repository/docker/winedepot/pinot) dockerHub repo.
+
+```SHELL
+./docker-push.sh winedepot/pinot:latest
+```
+
+* Tag a built image, then push.
+````SHELL
+docker tag pinot:release-0.1.0 winedepot/pinot:release-0.1.0
+docker push winedepot/pinot:release-0.1.0
+````
+
+Script `docker-build-and-push.sh` builds and publishes this docker image to your docker registry after build.
+
+* Example of building and publishing a image to [winedepot/pinot](https://cloud.docker.com/u/winedepot/repository/docker/winedepot/pinot) dockerHub repo.
+
+```SHELL
+./docker-build-and-push.sh winedepot/pinot:latest master https://github.com/apache/incubator-pinot.git
+```
+
+## How to Run it
+
+The entry point of docker image is `pinot-admin.sh` script.
+
+### Bring up Zookeeper
+Example of bring up a local zookeeper in docker:
+```SHELL
+docker pull zookeeper
+docker run --name pinot-zookeeper --restart always -p 2181:2181 zookeeper
+```
+You can extract the zookeeper host from:
+```SHELL
+docker inspect pinot-zookeeper|grep IPAddress
+ "SecondaryIPAddresses": null,
+ "IPAddress": "172.17.0.2",
+ "IPAddress": "172.17.0.2",
+```
+Please use local zookeeper path `172.17.0.2:2181` as `-zkAddress` parameter.
+
+### Pinot Controller
+Example of bring up a local controller:
+```SHELL
+docker run -p 9000:9000 pinot:release-0.1.0 StartController -zkAddress 172.17.0.2:2181
+```
+
+### Pinot Broker
+Example of bring up a local broker:
+```SHELL
+docker run -p 8099:8099 pinot:release-0.1.0 StartBroker -zkAddress 172.17.0.2:2181
+```
+
+### Pinot Server
+Example of bring up a local server:
+```SHELL
+docker run -p 8098:8098 pinot:release-0.1.0 StartServer -zkAddress 172.17.0.2:2181
+```
\ No newline at end of file
diff --git a/docker/docker-build-and-push.sh b/docker/docker-build-and-push.sh
new file mode 100755
index 0000000..c95f2de
--- /dev/null
+++ b/docker/docker-build-and-push.sh
@@ -0,0 +1,30 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+
+sh docker-build.sh $@
+if [[ "$#" -gt 0 ]]
+then
+ DOCKER_TAG=$1
+else
+ DOCKER_TAG="pinot:latest"
+fi
+echo "Trying to push docker image to ${DOCKER_TAG}"
+docker push ${DOCKER_TAG}
diff --git a/docker/docker-build.sh b/docker/docker-build.sh
new file mode 100755
index 0000000..9fda3e5
--- /dev/null
+++ b/docker/docker-build.sh
@@ -0,0 +1,47 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+
+if [[ "$#" -gt 0 ]]
+then
+ DOCKER_TAG=$1
+else
+ DOCKER_TAG="pinot:latest"
+ echo "Not specified a Docker Tag, using default tag: ${DOCKER_TAG}."
+fi
+
+if [[ "$#" -gt 1 ]]
+then
+ PINOT_BRANCH=$2
+else
+ PINOT_BRANCH=master
+ echo "Not specified a Pinot branch to build, using default branch: ${PINOT_BRANCH}."
+fi
+
+if [[ "$#" -gt 2 ]]
+then
+ PINOT_GIT_URL=$3
+else
+ PINOT_GIT_URL="https://github.com/apache/incubator-pinot.git"
+fi
+
+echo "Trying to build Pinot docker image from Git URL: [ ${PINOT_GIT_URL} ] on branch: [ ${PINOT_BRANCH} ] and tag it as: [ ${DOCKER_TAG} ]"
+
+docker build --no-cache -t ${DOCKER_TAG} --build-arg PINOT_BRANCH=${PINOT_BRANCH} --build-arg PINOT_GIT_URL=${PINOT_GIT_URL} -f Dockerfile .
diff --git a/docker/docker-push.sh b/docker/docker-push.sh
new file mode 100755
index 0000000..cfef8fe
--- /dev/null
+++ b/docker/docker-push.sh
@@ -0,0 +1,29 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+
+if [[ "$#" -gt 0 ]]
+then
+ DOCKER_TAG=$1
+else
+ DOCKER_TAG="pinot:latest"
+fi
+echo "Trying to push docker image to ${DOCKER_TAG}"
+docker push ${DOCKER_TAG}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org