You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by ji...@apache.org on 2018/10/28 05:33:26 UTC
[mesos] branch 1.7.x updated (6855c75 -> cb07b69)
This is an automated email from the ASF dual-hosted git repository.
jieyu pushed a change to branch 1.7.x
in repository https://gitbox.apache.org/repos/asf/mesos.git.
from 6855c75 Added MESOS-9305 to 1.7.1 CHANGELOG.
new e2ca418 Refactored the CentOS RPM and Docker build.
new f267be6 Updated Mesos-mini scripts.
new 36e49a3 Updated Jenkins Docker Centos script.
new 5c27823 Updated Jenkins Mesos mini script.
new ff806b7 Added Jenkinsfile for docker centos.
new 2d58fd2 Fixed a typo in Docker centos Jenkinsfile.
new e59b555 Added Jenkinsfile for Mesos mini.
new 4a41ffc Added Jenkinsfile for building RPMs.
new d050bc1 Used a better default for RPM builder docker image name.
new cb07b69 Removed CentOS6 build from Jenkins pipeline.
The 10 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
support/jenkins/Jenkinsfile-docker-centos | 17 ++++++
support/jenkins/Jenkinsfile-mini | 17 ++++++
support/jenkins/Jenkinsfile-packaging-centos | 17 ++++++
support/jenkins/docker-centos.sh | 34 ++++++-----
support/jenkins/mini.sh | 45 +++++++--------
support/mesos-mini/Dockerfile | 4 +-
support/mesos-mini/build.sh | 17 +++---
support/mesos-mini/docker_env.sh | 2 +-
support/mesos-mini/mesos/agent_environment | 1 -
.../mesos-mini/mesos/modules/uri_disk_profile.json | 2 +-
support/packaging/centos/build-docker-centos.sh | 34 +++++++++++
support/packaging/centos/build-docker-image.sh | 57 -------------------
support/packaging/centos/build-docker-rpmbuild.sh | 33 +++++++++++
support/packaging/centos/build-rpm-docker.sh | 66 ++++++++++++++++++++++
support/packaging/centos/build_rpm.sh | 34 +++++++----
support/packaging/centos/centos6.dockerfile | 9 +++
support/packaging/centos/centos7.dockerfile | 9 +++
support/packaging/centos/user-init.sh | 18 ++++++
18 files changed, 295 insertions(+), 121 deletions(-)
create mode 100644 support/jenkins/Jenkinsfile-docker-centos
create mode 100644 support/jenkins/Jenkinsfile-mini
create mode 100644 support/jenkins/Jenkinsfile-packaging-centos
create mode 100755 support/packaging/centos/build-docker-centos.sh
delete mode 100755 support/packaging/centos/build-docker-image.sh
create mode 100755 support/packaging/centos/build-docker-rpmbuild.sh
create mode 100755 support/packaging/centos/build-rpm-docker.sh
create mode 100755 support/packaging/centos/user-init.sh
[mesos] 10/10: Removed CentOS6 build from Jenkins pipeline.
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jieyu pushed a commit to branch 1.7.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit cb07b697d4fbd8143136b39dd48a64dd634c2c4f
Author: Jie Yu <yu...@gmail.com>
AuthorDate: Sat Oct 27 22:20:15 2018 -0700
Removed CentOS6 build from Jenkins pipeline.
Due to the deprecation of devtoolset-3-gcc:
https://www.centos.org/forums/viewtopic.php?f=14&t=67978
This does not build anymore. Disable it from the pipeline for the moment
while we're figuring out if we want to continue support it or not.
(cherry picked from commit 8d7c7c9ffef65c01182a4370fd4a1f32ac3ada3d)
---
support/jenkins/Jenkinsfile-packaging-centos | 9 ---------
1 file changed, 9 deletions(-)
diff --git a/support/jenkins/Jenkinsfile-packaging-centos b/support/jenkins/Jenkinsfile-packaging-centos
index 0ca59b9..3d87e03 100644
--- a/support/jenkins/Jenkinsfile-packaging-centos
+++ b/support/jenkins/Jenkinsfile-packaging-centos
@@ -13,14 +13,5 @@ parallel(
archiveArtifacts(artifacts: '**/*.rpm', fingerprint: true)
}
- },
- centos6: {
- node(env.NODE_LABELS) {
- checkout scm
-
- sh("CENTOS_DISTRO=6 support/packaging/centos/build-rpm-docker.sh")
-
- archiveArtifacts(artifacts: '**/*.rpm', fingerprint: true)
- }
}
)
[mesos] 03/10: Updated Jenkins Docker Centos script.
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jieyu pushed a commit to branch 1.7.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit 36e49a39867bae96a34a6a014c5d0d3d0ee66120
Author: Jie Yu <yu...@gmail.com>
AuthorDate: Sat Oct 27 11:48:08 2018 -0700
Updated Jenkins Docker Centos script.
To adapt to the new Docker Centos build pipeline.
(cherry picked from commit e8602b6a2c816aac2853ea10e28fcf5e36879076)
---
support/jenkins/docker-centos.sh | 34 ++++++++++++++++++++--------------
1 file changed, 20 insertions(+), 14 deletions(-)
diff --git a/support/jenkins/docker-centos.sh b/support/jenkins/docker-centos.sh
index 812799a..12cf81c 100755
--- a/support/jenkins/docker-centos.sh
+++ b/support/jenkins/docker-centos.sh
@@ -8,26 +8,32 @@ set -o errexit -o nounset -o pipefail -o verbose
CURRENT_DIR="$(cd "$(dirname "$0")"; pwd -P)"
SUPPORT_DIR="${CURRENT_DIR}/.."
-: ${USERNAME:?"Environment variable 'USERNAME' must be set to the username of the 'Mesos DockerBot' Docker hub account."}
-: ${PASSWORD:?"Environment variable 'PASSWORD' must be set to the password of the 'Mesos DockerBot' Docker hub account."}
+: "${USERNAME:?"Environment variable 'USERNAME' must be set to the username of the 'Mesos DockerBot' Docker hub account."}"
+: "${PASSWORD:?"Environment variable 'PASSWORD' must be set to the password of the 'Mesos DockerBot' Docker hub account."}"
-DOCKER_IMAGE_PACKAGING=${DOCKER_IMAGE_PACKAGING:-"mesos/mesos-centos-packaging"}
-DOCKER_IMAGE_DISTRO=${DOCKER_IMAGE_DISTRO:-"mesos/mesos-centos"}
-DOCKER_IMAGE_TAG=`date +%F`
+DOCKER_IMAGE=${DOCKER_IMAGE:-"mesos/mesos-centos"}
+
+MESOS_SHA=${MESOS_SHA:-$(git rev-parse HEAD)}
+MESOS_TAG_OR_SHA=$(git describe --exact-match "${MESOS_SHA}" 2>/dev/null || echo "${MESOS_SHA}")
+DOCKER_IMAGE_TAG="${MESOS_TAG_OR_SHA}"
+
+echo "MESOS_SHA=${MESOS_SHA}"
+echo "DOCKER_IMAGE_TAG=${DOCKER_IMAGE_TAG}"
function cleanup {
- docker rmi $(docker images -q ${DOCKER_IMAGE_PACKAGING}:${DOCKER_IMAGE_TAG}) || true
- docker rmi $(docker images -q ${DOCKER_IMAGE_DISTRO}:${DOCKER_IMAGE_TAG}) || true
+ docker rmi "$(docker images -q "${DOCKER_IMAGE}:${DOCKER_IMAGE_TAG}")" || true
}
trap cleanup EXIT
-DOCKER_IMAGE_PACKAGING=${DOCKER_IMAGE_PACKAGING} \
-DOCKER_IMAGE_DISTRO=${DOCKER_IMAGE_DISTRO} \
+CENTOS_DISTRO="7" \
+DOCKER_IMAGE=${DOCKER_IMAGE} \
DOCKER_IMAGE_TAG=${DOCKER_IMAGE_TAG} \
-"${SUPPORT_DIR}/packaging/centos/build-docker-image.sh"
+"${SUPPORT_DIR}/packaging/centos/build-docker-centos.sh"
+
+DATE=$(date +%F)
-docker tag ${DOCKER_IMAGE_DISTRO}:${DOCKER_IMAGE_TAG} ${DOCKER_IMAGE_DISTRO}:latest
-docker login -u ${USERNAME} -p ${PASSWORD}
-docker push ${DOCKER_IMAGE_DISTRO}:${DOCKER_IMAGE_TAG}
-docker push ${DOCKER_IMAGE_DISTRO}:latest
+docker tag "${DOCKER_IMAGE}:${DOCKER_IMAGE_TAG}" "${DOCKER_IMAGE}:${DATE}"
+docker login -u "${USERNAME}" -p "${PASSWORD}"
+docker push "${DOCKER_IMAGE}:${DOCKER_IMAGE_TAG}"
+docker push "${DOCKER_IMAGE}:${DATE}"
[mesos] 01/10: Refactored the CentOS RPM and Docker build.
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jieyu pushed a commit to branch 1.7.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit e2ca4186918c6519f560ebc340e4fd9c23b118ea
Author: Jie Yu <yu...@gmail.com>
AuthorDate: Sat Oct 27 11:44:12 2018 -0700
Refactored the CentOS RPM and Docker build.
This patch enables the build of RPM packages using Docker so that you
can build RPM from anywhere (e.g., your Macbook).
This patch also fixes an user/group related issue with Docker which
might cause permission denided failures during cleanup. The idea is to
add the expected user to the docker container that's being used to build
RPM.
(cherry picked from commit efb6b884242eb0ccecf10982d4b407356924a35e)
---
support/packaging/centos/build-docker-centos.sh | 34 ++++++++++++
support/packaging/centos/build-docker-image.sh | 57 --------------------
support/packaging/centos/build-docker-rpmbuild.sh | 33 ++++++++++++
support/packaging/centos/build-rpm-docker.sh | 66 +++++++++++++++++++++++
support/packaging/centos/build_rpm.sh | 34 ++++++++----
support/packaging/centos/centos6.dockerfile | 9 ++++
support/packaging/centos/centos7.dockerfile | 9 ++++
support/packaging/centos/user-init.sh | 18 +++++++
8 files changed, 192 insertions(+), 68 deletions(-)
diff --git a/support/packaging/centos/build-docker-centos.sh b/support/packaging/centos/build-docker-centos.sh
new file mode 100755
index 0000000..d6d3d99
--- /dev/null
+++ b/support/packaging/centos/build-docker-centos.sh
@@ -0,0 +1,34 @@
+#!/usr/bin/env bash
+
+set -o errexit -o nounset -o pipefail -o verbose
+
+# This script builds a CentOS based docker image with Mesos installed
+# using the current head of the source tree.
+
+CENTOS_DIR="$(cd "$(dirname "$0")"; pwd -P)"
+SOURCE_DIR="$(cd "${CENTOS_DIR}/../../.."; pwd -P)"
+
+CENTOS_DISTRO=${CENTOS_DISTRO:-"7"}
+DOCKER_IMAGE=${DOCKER_IMAGE:-"mesos/mesos-centos"}
+DOCKER_IMAGE_TAG=${DOCKER_IMAGE_TAG:-"latest"}
+
+if ! [ -x "$(command -v docker)" ]; then
+ echo 'Error: docker is not installed.' >&2
+ exit 1
+fi
+
+"${CENTOS_DIR}/build-rpm-docker.sh"
+
+# Build the image for running Mesos.
+DOCKER_CONTEXT_DIR="${SOURCE_DIR}/centos${CENTOS_DISTRO}/rpmbuild/RPMS/x86_64"
+
+cat <<EOF > "${DOCKER_CONTEXT_DIR}/Dockerfile"
+FROM centos:${CENTOS_DISTRO}
+ADD mesos-?.?.?-*.rpm /
+RUN yum --nogpgcheck -y localinstall /mesos-*.rpm
+EOF
+
+docker build \
+ --rm \
+ -t "${DOCKER_IMAGE}:${DOCKER_IMAGE_TAG}" \
+ "${DOCKER_CONTEXT_DIR}"
diff --git a/support/packaging/centos/build-docker-image.sh b/support/packaging/centos/build-docker-image.sh
deleted file mode 100755
index cf7c298..0000000
--- a/support/packaging/centos/build-docker-image.sh
+++ /dev/null
@@ -1,57 +0,0 @@
-#!/usr/bin/env bash
-
-set -o errexit -o nounset -o pipefail -o verbose
-
-# This script builds a CentOS based docker image with Mesos installed
-# using the current head of the source tree.
-
-CENTOS_DIR="$(cd "$(dirname "$0")"; pwd -P)"
-SOURCE_DIR="$(cd ${CENTOS_DIR}/../../..; pwd -P)"
-
-DOCKER_IMAGE_PACKAGING=${DOCKER_IMAGE_PACKAGING:-"mesos/mesos-centos-packaging"}
-DOCKER_IMAGE_DISTRO=${DOCKER_IMAGE_DISTRO:-"mesos/mesos-centos"}
-DOCKER_IMAGE_TAG=${DOCKER_IMAGE_TAG:-"latest"}
-
-if ! [ -x "$(command -v docker)" ]; then
- echo 'Error: docker is not installed.' >&2
- exit 1
-fi
-
-function cleanup {
- rm -rf "${SOURCE_DIR}/centos7"
-}
-
-cleanup
-
-trap cleanup EXIT
-
-# Build the image for building Mesos packages.
-docker build \
- --rm \
- -t ${DOCKER_IMAGE_PACKAGING}:${DOCKER_IMAGE_TAG} \
- -f "${SOURCE_DIR}/support/packaging/centos/centos7.dockerfile" \
- "${SOURCE_DIR}/support/packaging/centos/"
-
-# Build the RPM.
-USER_ID=`id -u`
-GROUP_ID=`id -g`
-
-docker run \
- --rm \
- -v "${SOURCE_DIR}:${SOURCE_DIR}" \
- ${DOCKER_IMAGE_PACKAGING}:${DOCKER_IMAGE_TAG} \
- /bin/bash -c "${SOURCE_DIR}/support/packaging/centos/build_rpm.sh && chown -R ${USER_ID}:${GROUP_ID} ${SOURCE_DIR}/centos7"
-
-# Build the image for running Mesos.
-DOCKER_CONTEXT_DIR="${SOURCE_DIR}/centos7/rpmbuild/RPMS/x86_64"
-
-cat <<EOF > "${DOCKER_CONTEXT_DIR}/Dockerfile"
-FROM centos:7
-ADD mesos-?.?.?-*.rpm /
-RUN yum --nogpgcheck -y localinstall /mesos-*.rpm
-EOF
-
-docker build \
- --rm \
- -t ${DOCKER_IMAGE_DISTRO}:${DOCKER_IMAGE_TAG} \
- "${DOCKER_CONTEXT_DIR}"
diff --git a/support/packaging/centos/build-docker-rpmbuild.sh b/support/packaging/centos/build-docker-rpmbuild.sh
new file mode 100755
index 0000000..1fdd5c7
--- /dev/null
+++ b/support/packaging/centos/build-docker-rpmbuild.sh
@@ -0,0 +1,33 @@
+#!/usr/bin/env bash
+
+set -o errexit -o nounset -o pipefail -o verbose
+
+# This script builds a Docker image that will be used to build RPM
+# package for Mesos.
+
+CENTOS_DIR="$(cd "$(dirname "$0")"; pwd -P)"
+
+CENTOS_DISTRO=${CENTOS_DISTRO:-"7"}
+DOCKER_IMAGE=${DOCKER_IMAGE:-"mesos/mesos-centos-rpmbuild"}
+DOCKER_IMAGE_TAG=${DOCKER_IMAGE_TAG:-"latest"}
+
+if ! [ -x "$(command -v docker)" ]; then
+ echo 'Error: docker is not installed.' >&2
+ exit 1
+fi
+
+USER_NAME=$(id -u -n)
+USER_ID=$(id -u)
+GROUP_NAME=$(id -g -n)
+GROUP_ID=$(id -g)
+
+# Build the image for building Mesos packages.
+docker build \
+ --rm \
+ --build-arg "USER_NAME=${USER_NAME}" \
+ --build-arg "USER_ID=${USER_ID}" \
+ --build-arg "GROUP_NAME=${GROUP_NAME}" \
+ --build-arg "GROUP_ID=${GROUP_ID}" \
+ -t "${DOCKER_IMAGE}:${DOCKER_IMAGE_TAG}" \
+ -f "${CENTOS_DIR}/centos${CENTOS_DISTRO}.dockerfile" \
+ "${CENTOS_DIR}"
diff --git a/support/packaging/centos/build-rpm-docker.sh b/support/packaging/centos/build-rpm-docker.sh
new file mode 100755
index 0000000..e7d9c3f
--- /dev/null
+++ b/support/packaging/centos/build-rpm-docker.sh
@@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+
+set -o errexit -o nounset -o pipefail -o verbose
+
+# This script builds RPM package for Mesos using Docker.
+
+CENTOS_DIR="$(cd "$(dirname "$0")"; pwd -P)"
+SOURCE_DIR="$(cd "${CENTOS_DIR}/../../.."; pwd -P)"
+
+CENTOS_DISTRO=${CENTOS_DISTRO:-"7"}
+DOCKER_IMAGE=${DOCKER_IMAGE:-"mesos/mesos-centos-rpmbuild"}
+DOCKER_IMAGE_TAG=${DOCKER_IMAGE_TAG:-"latest"}
+
+if ! [ -x "$(command -v docker)" ]; then
+ echo 'Error: docker is not installed.' >&2
+ exit 1
+fi
+
+MESOS_SHA=${MESOS_SHA:-$(git rev-parse HEAD)}
+MESOS_TAG_OR_SHA=$(git describe --exact-match "${MESOS_SHA}" 2>/dev/null || echo "${MESOS_SHA}")
+
+if [[ "${MESOS_TAG_OR_SHA}" != "${MESOS_SHA}" ]]; then
+ # HEAD is also a tag.
+ MESOS_TAG="${MESOS_TAG_OR_SHA}"
+else
+ MESOS_TAG=""
+fi
+
+echo "MESOS_SHA=${MESOS_SHA}"
+echo "MESOS_TAG=${MESOS_TAG}"
+
+rm -rf "${SOURCE_DIR}/centos${CENTOS_DISTRO}"
+
+CENTOS_DISTRO="${CENTOS_DISTRO}" \
+DOCKER_IMAGE="${DOCKER_IMAGE}" \
+DOCKER_IMAGE_TAG="${DOCKER_IMAGE_TAG}" \
+"${CENTOS_DIR}/build-docker-rpmbuild.sh"
+
+USER_ID=$(id -u)
+GROUP_ID=$(id -g)
+
+# NOTE: A shared volume has the same gid as its host volume on Linux,
+# but the same group name on macOS. To run this script on both
+# platforms, we run the build with the group name of the docker
+# socket.
+DOCKER_SOCKET_GID=$(echo /var/run/docker.sock | perl -lne 'use File::stat; print stat($_)->gid')
+
+# Attach to terminal if we have a TTY so that things like CTRL-C work.
+if [ -t 1 ]; then
+ TTYARGS="-ti"
+else
+ TTYARGS=""
+fi
+
+docker run \
+ $TTYARGS \
+ --rm \
+ --user "${USER_ID}:${GROUP_ID}" \
+ --group-add 0 \
+ --group-add "${DOCKER_SOCKET_GID}" \
+ -v "${SOURCE_DIR}:${SOURCE_DIR}" \
+ "${DOCKER_IMAGE}:${DOCKER_IMAGE_TAG}" \
+ /bin/bash -c "MAKE_DIST=true MESOS_TAG=${MESOS_TAG} ${CENTOS_DIR}/build_rpm.sh"
+
+echo "RPM has been built and can be found at:"
+echo "${SOURCE_DIR}/centos${CENTOS_DISTRO}/rpmbuild/RPMS/x86_64"
diff --git a/support/packaging/centos/build_rpm.sh b/support/packaging/centos/build_rpm.sh
index abb6d4d..f3d83b6 100755
--- a/support/packaging/centos/build_rpm.sh
+++ b/support/packaging/centos/build_rpm.sh
@@ -19,11 +19,7 @@ if [ "$CENTOS_VERSION" = "6" ]; then
source scl_source enable devtoolset-3
fi
-if [ -z "$MESOS_TAG" ]; then
- gitsha=$(git rev-parse --short HEAD)
- snapshot_version=$(date -u +'%Y%m%d')git$gitsha
- MESOS_RELEASE=0.1.pre.$snapshot_version
-
+make_dist() {
pushd $MESOS_DIR
./bootstrap
popd
@@ -36,14 +32,30 @@ if [ -z "$MESOS_TAG" ]; then
cp -f $TMP_BUILD_DIR/mesos-$MESOS_VERSION.tar.gz $HOME/rpmbuild/SOURCES/
rm -rf $TMP_BUILD_DIR
+}
+
+if [ -z "$MESOS_TAG" ]; then
+ gitsha=$(git rev-parse --short HEAD)
+ snapshot_version=$(date -u +'%Y%m%d')git$gitsha
+ MESOS_RELEASE=0.1.pre.$snapshot_version
+
+ make_dist
elif [ "$MESOS_VERSION" = "$MESOS_TAG" ]; then
- curl -sSL \
- https://dist.apache.org/repos/dist/release/mesos/${MESOS_VERSION}/mesos-${MESOS_VERSION}.tar.gz \
- -o $HOME/rpmbuild/SOURCES/mesos-${MESOS_VERSION}.tar.gz
+ if [ ! -z ${MAKE_DIST:-""} ]; then
+ make_dist
+ else
+ curl -sSL \
+ https://dist.apache.org/repos/dist/release/mesos/${MESOS_VERSION}/mesos-${MESOS_VERSION}.tar.gz \
+ -o $HOME/rpmbuild/SOURCES/mesos-${MESOS_VERSION}.tar.gz
+ fi
else
- curl -sSL \
- https://dist.apache.org/repos/dist/dev/mesos/${MESOS_TAG}/mesos-${MESOS_VERSION}.tar.gz \
- -o $HOME/rpmbuild/SOURCES/mesos-${MESOS_VERSION}.tar.gz
+ if [ ! -z ${MAKE_DIST:-""} ]; then
+ make_dist
+ else
+ curl -sSL \
+ https://dist.apache.org/repos/dist/dev/mesos/${MESOS_TAG}/mesos-${MESOS_VERSION}.tar.gz \
+ -o $HOME/rpmbuild/SOURCES/mesos-${MESOS_VERSION}.tar.gz
+ fi
fi
rpmbuild \
diff --git a/support/packaging/centos/centos6.dockerfile b/support/packaging/centos/centos6.dockerfile
index ec13b24..457f48d 100644
--- a/support/packaging/centos/centos6.dockerfile
+++ b/support/packaging/centos/centos6.dockerfile
@@ -36,3 +36,12 @@ ADD mesos.spec /mesos.spec
RUN yum makecache && \
yum-builddep -y /mesos.spec
+
+ADD user-init.sh /user-init.sh
+
+ARG USER_NAME=root
+ARG USER_ID=0
+ARG GROUP_NAME=root
+ARG GROUP_ID=0
+
+RUN /user-init.sh $USER_NAME $USER_ID $GROUP_NAME $GROUP_ID
diff --git a/support/packaging/centos/centos7.dockerfile b/support/packaging/centos/centos7.dockerfile
index 95b4d28..b69cf3e 100644
--- a/support/packaging/centos/centos7.dockerfile
+++ b/support/packaging/centos/centos7.dockerfile
@@ -33,3 +33,12 @@ ADD mesos.spec /mesos.spec
RUN yum makecache && \
yum-builddep -y /mesos.spec
+
+ADD user-init.sh /user-init.sh
+
+ARG USER_NAME=root
+ARG USER_ID=0
+ARG GROUP_NAME=root
+ARG GROUP_ID=0
+
+RUN /user-init.sh $USER_NAME $USER_ID $GROUP_NAME $GROUP_ID
diff --git a/support/packaging/centos/user-init.sh b/support/packaging/centos/user-init.sh
new file mode 100755
index 0000000..9c679a1
--- /dev/null
+++ b/support/packaging/centos/user-init.sh
@@ -0,0 +1,18 @@
+#!/usr/bin/env bash
+
+set -o errexit -o nounset -o pipefail -o verbose
+
+USER_NAME=$1
+USER_ID=$2
+GROUP_NAME=$3
+GROUP_ID=$4
+
+# Create the group.
+if ! getent group "${GROUP_ID}"; then
+ groupadd -r -g "${GROUP_ID}" "${GROUP_NAME}"
+fi
+
+# Create the user.
+if ! getent passwd "${USER_ID}"; then
+ useradd -r -m -u "${USER_ID}" "${USER_NAME}" -g "${GROUP_ID}"
+fi
[mesos] 02/10: Updated Mesos-mini scripts.
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jieyu pushed a commit to branch 1.7.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit f267be6abf3747d609969f1bd955be3e5e1af2f1
Author: Jie Yu <yu...@gmail.com>
AuthorDate: Sat Oct 27 11:47:06 2018 -0700
Updated Mesos-mini scripts.
This patch fixed some issues in the Mesos-mini build scripts and adapted
to the new centos Docker build workflow.
(cherry picked from commit 20ed47b839b2713912b02770a381a9e000b52ea8)
---
support/mesos-mini/Dockerfile | 4 +++-
support/mesos-mini/build.sh | 17 ++++++++---------
support/mesos-mini/docker_env.sh | 2 +-
support/mesos-mini/mesos/agent_environment | 1 -
support/mesos-mini/mesos/modules/uri_disk_profile.json | 2 +-
5 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/support/mesos-mini/Dockerfile b/support/mesos-mini/Dockerfile
index f317359..94b0836 100644
--- a/support/mesos-mini/Dockerfile
+++ b/support/mesos-mini/Dockerfile
@@ -1,4 +1,6 @@
-FROM mesos/mesos-centos
+ARG BASE_IMAGE_NAME
+
+FROM $BASE_IMAGE_NAME
RUN yum install -y java-1.8.0-openjdk iptables
diff --git a/support/mesos-mini/build.sh b/support/mesos-mini/build.sh
index dc9c2bb..eb9d187 100755
--- a/support/mesos-mini/build.sh
+++ b/support/mesos-mini/build.sh
@@ -8,20 +8,19 @@ set -o errexit -o nounset -o pipefail -o verbose
CURRENT_DIR="$(cd "$(dirname "$0")"; pwd -P)"
SUPPORT_DIR="${CURRENT_DIR}/.."
-DOCKER_IMAGE_MINI=${DOCKER_IMAGE_MINI:-"mesos/mesos-mini"}
-DOCKER_IMAGE_PACKAGING=${DOCKER_IMAGE_PACKAGING:-"mesos/mesos-centos-packaging"}
-DOCKER_IMAGE_DISTRO=${DOCKER_IMAGE_DISTRO:-"mesos/mesos-centos"}
+DOCKER_IMAGE=${DOCKER_IMAGE_MINI:-"mesos/mesos-mini"}
DOCKER_IMAGE_TAG=${DOCKER_IMAGE_TAG:-"latest"}
+DOCKER_IMAGE_DISTRO=${DOCKER_IMAGE_DISTRO:-"mesos/mesos-centos"}
if ! [ -x "$(command -v docker)" ]; then
echo 'Error: docker is not installed.' >&2
exit 1
fi
-DOCKER_IMAGE_PACKAGING=${DOCKER_IMAGE_PACKAGING} \
-DOCKER_IMAGE_DISTRO=${DOCKER_IMAGE_DISTRO} \
-DOCKER_IMAGE_TAG=${DOCKER_IMAGE_TAG} \
-"${SUPPORT_DIR}/packaging/centos/build-docker-image.sh"
+DOCKER_IMAGE=${DOCKER_IMAGE_DISTRO} \
+"${SUPPORT_DIR}/packaging/centos/build-docker-centos.sh"
-docker tag ${DOCKER_IMAGE_DISTRO}:${DOCKER_IMAGE_TAG} "mesos/mesos-centos"
-docker build -t ${DOCKER_IMAGE_MINI}:${DOCKER_IMAGE_TAG} "${CURRENT_DIR}/"
+docker build \
+ --build-arg "BASE_IMAGE_NAME=${DOCKER_IMAGE_DISTRO}" \
+ -t "${DOCKER_IMAGE}:${DOCKER_IMAGE_TAG}" \
+ "${CURRENT_DIR}/"
diff --git a/support/mesos-mini/docker_env.sh b/support/mesos-mini/docker_env.sh
index ee7db85..8d89dd4 100755
--- a/support/mesos-mini/docker_env.sh
+++ b/support/mesos-mini/docker_env.sh
@@ -2,7 +2,7 @@
set -o errexit -o nounset -o pipefail -o verbose
-CGROUP=`grep memory /proc/1/cgroup | cut -d: -f3`
+CGROUP=$(grep memory /proc/1/cgroup | cut -d: -f3)
cat <<EOF > "/etc/docker/env"
CGROUP_PARENT=${CGROUP}/docker
diff --git a/support/mesos-mini/mesos/agent_environment b/support/mesos-mini/mesos/agent_environment
index c0dccae99..4876728 100644
--- a/support/mesos-mini/mesos/agent_environment
+++ b/support/mesos-mini/mesos/agent_environment
@@ -11,7 +11,6 @@ MESOS_IMAGE_PROVISIONER_BACKEND=copy
MESOS_NETWORK_CNI_CONFIG_DIR=/etc/mesos/cni
MESOS_NETWORK_CNI_PLUGINS_DIR=/usr/libexec/mesos/cni/:/usr/libexec/mesos/
MESOS_SYSTEMD_ENABLE_SUPPORT=false
-MESOS_AGENT_FEATURES=file:///etc/mesos/agent_features.json
MESOS_RESOURCE_PROVIDER_CONFIG_DIR=/etc/mesos/resource_providers
MESOS_DISK_PROFILE_ADAPTOR=org_apache_mesos_UriDiskProfileAdaptor
MESOS_MODULES_DIR=/etc/mesos/modules
diff --git a/support/mesos-mini/mesos/modules/uri_disk_profile.json b/support/mesos-mini/mesos/modules/uri_disk_profile.json
index 53bdded..ab2061a 100644
--- a/support/mesos-mini/mesos/modules/uri_disk_profile.json
+++ b/support/mesos-mini/mesos/modules/uri_disk_profile.json
@@ -1,7 +1,7 @@
{
"libraries": [
{
- "file": "/usr/lib64/mesos/modules/liburi_disk_profile.so",
+ "file": "/usr/lib64/mesos/modules/liburi_disk_profile_adaptor.so",
"modules": [
{
"name": "org_apache_mesos_UriDiskProfileAdaptor",
[mesos] 04/10: Updated Jenkins Mesos mini script.
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jieyu pushed a commit to branch 1.7.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit 5c2782334029717346c1a9d6e627bf3c87665a87
Author: Jie Yu <yu...@gmail.com>
AuthorDate: Sat Oct 27 13:03:41 2018 -0700
Updated Jenkins Mesos mini script.
To adapt to the new Docker Centos build pipeline.
(cherry picked from commit 8583c5e44536dacc60225d5662ac63151466f6d7)
---
support/jenkins/mini.sh | 45 +++++++++++++++++++--------------------------
1 file changed, 19 insertions(+), 26 deletions(-)
diff --git a/support/jenkins/mini.sh b/support/jenkins/mini.sh
index 4f94b6c..76b3cc7 100755
--- a/support/jenkins/mini.sh
+++ b/support/jenkins/mini.sh
@@ -5,41 +5,34 @@ set -o errexit -o nounset -o pipefail -o verbose
CURRENT_DIR="$(cd "$(dirname "$0")"; pwd -P)"
SUPPORT_DIR="${CURRENT_DIR}/.."
-: ${USERNAME:?"Environment variable 'USERNAME' must be set to the username of the 'Mesos DockerBot' Docker hub account."}
-: ${PASSWORD:?"Environment variable 'PASSWORD' must be set to the password of the 'Mesos DockerBot' Docker hub account."}
+: "${USERNAME:?"Environment variable 'USERNAME' must be set to the username of the 'Mesos DockerBot' Docker hub account."}"
+: "${PASSWORD:?"Environment variable 'PASSWORD' must be set to the password of the 'Mesos DockerBot' Docker hub account."}"
-MESOS_SHA=${MESOS_SHA:-`git rev-parse HEAD`}
-MESOS_VERSION=`git describe --exact-match $MESOS_SHA 2>/dev/null || echo $MESOS_SHA`
-
-echo "MESOS_SHA=$MESOS_SHA"
-echo "MESOS_VERSION=$MESOS_VERSION"
-
-DOCKER_IMAGE_MINI=${DOCKER_IMAGE_MINI:-"mesos/mesos-mini"}
-DOCKER_IMAGE_PACKAGING=${DOCKER_IMAGE_PACKAGING:-"mesos/mesos-centos-packaging"}
+DOCKER_IMAGE=${DOCKER_IMAGE:-"mesos/mesos-mini"}
DOCKER_IMAGE_DISTRO=${DOCKER_IMAGE_DISTRO:-"mesos/mesos-centos"}
-if [[ "$MESOS_VERSION" != "$MESOS_SHA" ]]; then
- # HEAD is also a tag.
- DOCKER_IMAGE_TAG="$MESOS_VERSION"
-else
- DOCKER_IMAGE_TAG=`date +%F`
-fi
+MESOS_SHA=${MESOS_SHA:-$(git rev-parse HEAD)}
+MESOS_TAG_OR_SHA=$(git describe --exact-match "${MESOS_SHA}" 2>/dev/null || echo "${MESOS_SHA}")
+DOCKER_IMAGE_TAG="${MESOS_TAG_OR_SHA}"
+
+echo "MESOS_SHA=${MESOS_SHA}"
+echo "DOCKER_IMAGE_TAG=${MESOS_TAG_OR_SHA}"
function cleanup {
- docker rmi $(docker images -q ${DOCKER_IMAGE_PACKAGING}:${DOCKER_IMAGE_TAG}) || true
- docker rmi $(docker images -q ${DOCKER_IMAGE_DISTRO}:${DOCKER_IMAGE_TAG}) || true
- docker rmi $(docker images -q ${DOCKER_IMAGE_MINI}:${DOCKER_IMAGE_TAG}) || true
+ docker rmi "$(docker images -q "${DOCKER_IMAGE}:${DOCKER_IMAGE_TAG}")" || true
+ docker rmi "$(docker images -q "${DOCKER_IMAGE_DISTRO}:${DOCKER_IMAGE_TAG}")" || true
}
trap cleanup EXIT
-DOCKER_IMAGE_MINI=${DOCKER_IMAGE_MINI} \
-DOCKER_IMAGE_PACKAGING=${DOCKER_IMAGE_PACKAGING} \
-DOCKER_IMAGE_DISTRO=${DOCKER_IMAGE_DISTRO} \
+DOCKER_IMAGE=${DOCKER_IMAGE} \
DOCKER_IMAGE_TAG=${DOCKER_IMAGE_TAG} \
+DOCKER_IMAGE_DISTRO=${DOCKER_IMAGE_DISTRO} \
"${SUPPORT_DIR}/mesos-mini/build.sh"
-docker tag ${DOCKER_IMAGE_MINI}:${DOCKER_IMAGE_TAG} ${DOCKER_IMAGE_MINI}:latest
-docker login -u ${USERNAME} -p ${PASSWORD}
-docker push ${DOCKER_IMAGE_MINI}:${DOCKER_IMAGE_TAG}
-docker push ${DOCKER_IMAGE_MINI}:latest
+DATE=$(date +%F)
+
+docker tag "${DOCKER_IMAGE}:${DOCKER_IMAGE_TAG}" "${DOCKER_IMAGE}:${DATE}"
+docker login -u "${USERNAME}" -p "${PASSWORD}"
+docker push "${DOCKER_IMAGE}:${DOCKER_IMAGE_TAG}"
+docker push "${DOCKER_IMAGE}:${DATE}"
[mesos] 08/10: Added Jenkinsfile for building RPMs.
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jieyu pushed a commit to branch 1.7.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit 4a41ffc0543c7984310c392663e3a4fb1cd5a7ea
Author: Jie Yu <yu...@gmail.com>
AuthorDate: Sat Oct 27 21:00:16 2018 -0700
Added Jenkinsfile for building RPMs.
(cherry picked from commit 2a27b473b996a28eebda639f2f4dc3e00c15a4bb)
---
support/jenkins/Jenkinsfile-packaging-centos | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/support/jenkins/Jenkinsfile-packaging-centos b/support/jenkins/Jenkinsfile-packaging-centos
new file mode 100644
index 0000000..0ca59b9
--- /dev/null
+++ b/support/jenkins/Jenkinsfile-packaging-centos
@@ -0,0 +1,26 @@
+#!groovy
+
+if (env.NODE_LABELS == null) {
+ env.NODE_LABELS = "(ubuntu)&&(!ubuntu-us1)&&(!ubuntu-eu2)&&(!qnode3)&&(!H23)"
+}
+
+parallel(
+ centos7: {
+ node(env.NODE_LABELS) {
+ checkout scm
+
+ sh("CENTOS_DISTRO=7 support/packaging/centos/build-rpm-docker.sh")
+
+ archiveArtifacts(artifacts: '**/*.rpm', fingerprint: true)
+ }
+ },
+ centos6: {
+ node(env.NODE_LABELS) {
+ checkout scm
+
+ sh("CENTOS_DISTRO=6 support/packaging/centos/build-rpm-docker.sh")
+
+ archiveArtifacts(artifacts: '**/*.rpm', fingerprint: true)
+ }
+ }
+)
[mesos] 05/10: Added Jenkinsfile for docker centos.
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jieyu pushed a commit to branch 1.7.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit ff806b720b1ae0cb349d8ce8a7dcbcc5966b63b0
Author: Jie Yu <yu...@gmail.com>
AuthorDate: Sat Oct 27 19:39:53 2018 -0700
Added Jenkinsfile for docker centos.
(cherry picked from commit 3fc568e2592a5623722563a9d6dba37f94390b7a)
---
support/jenkins/Jenkinsfile-docker-centos | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/support/jenkins/Jenkinsfile-docker-centos b/support/jenkins/Jenkinsfile-docker-centos
new file mode 100644
index 0000000..762462f
--- /dev/null
+++ b/support/jenkins/Jenkinsfile-docker-centos
@@ -0,0 +1,17 @@
+#!groovy
+
+if (env.NODE_LABELS == null) {
+ env.NODE_LABELS = "(ubuntu)&&(!ubuntu-us1)&&(!ubuntu-eu2)&&(!qnode3)&&(!H23)"
+}
+
+node(env.NODE_LABELS) {
+ checkout scm
+
+ withCredentials([[
+ $class: 'UsernamePasswordMultiBinding',
+ credentialsId: 'mesos-docker-bot'
+ usernameVariable: 'USERNAME',
+ passwordVariable: 'PASSWORD']]) {
+ sh("support/jenkins/docker-centos.sh")
+ }
+}
[mesos] 09/10: Used a better default for RPM builder docker image
name.
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jieyu pushed a commit to branch 1.7.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit d050bc1cae4b5ae9c2c6e658583c59cb7c231bc2
Author: Jie Yu <yu...@gmail.com>
AuthorDate: Sat Oct 27 22:19:32 2018 -0700
Used a better default for RPM builder docker image name.
(cherry picked from commit dd87797e88763677e45f4b7ad5f52438cd81cc83)
---
support/packaging/centos/build-docker-rpmbuild.sh | 2 +-
support/packaging/centos/build-rpm-docker.sh | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/support/packaging/centos/build-docker-rpmbuild.sh b/support/packaging/centos/build-docker-rpmbuild.sh
index 1fdd5c7..9c01aaf 100755
--- a/support/packaging/centos/build-docker-rpmbuild.sh
+++ b/support/packaging/centos/build-docker-rpmbuild.sh
@@ -8,7 +8,7 @@ set -o errexit -o nounset -o pipefail -o verbose
CENTOS_DIR="$(cd "$(dirname "$0")"; pwd -P)"
CENTOS_DISTRO=${CENTOS_DISTRO:-"7"}
-DOCKER_IMAGE=${DOCKER_IMAGE:-"mesos/mesos-centos-rpmbuild"}
+DOCKER_IMAGE=${DOCKER_IMAGE:-"mesos/mesos-centos${CENTOS_DISTRO}-rpmbuild"}
DOCKER_IMAGE_TAG=${DOCKER_IMAGE_TAG:-"latest"}
if ! [ -x "$(command -v docker)" ]; then
diff --git a/support/packaging/centos/build-rpm-docker.sh b/support/packaging/centos/build-rpm-docker.sh
index e7d9c3f..9bad6d8 100755
--- a/support/packaging/centos/build-rpm-docker.sh
+++ b/support/packaging/centos/build-rpm-docker.sh
@@ -8,7 +8,7 @@ CENTOS_DIR="$(cd "$(dirname "$0")"; pwd -P)"
SOURCE_DIR="$(cd "${CENTOS_DIR}/../../.."; pwd -P)"
CENTOS_DISTRO=${CENTOS_DISTRO:-"7"}
-DOCKER_IMAGE=${DOCKER_IMAGE:-"mesos/mesos-centos-rpmbuild"}
+DOCKER_IMAGE=${DOCKER_IMAGE:-"mesos/mesos-centos${CENTOS_DISTRO}-rpmbuild"}
DOCKER_IMAGE_TAG=${DOCKER_IMAGE_TAG:-"latest"}
if ! [ -x "$(command -v docker)" ]; then
[mesos] 06/10: Fixed a typo in Docker centos Jenkinsfile.
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jieyu pushed a commit to branch 1.7.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit 2d58fd27660975fde543d1426fc09e486ac1c662
Author: Jie Yu <yu...@gmail.com>
AuthorDate: Sat Oct 27 19:53:53 2018 -0700
Fixed a typo in Docker centos Jenkinsfile.
(cherry picked from commit 1ef77b767a0fed8a7d63c524167bd23311ef2601)
---
support/jenkins/Jenkinsfile-docker-centos | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/support/jenkins/Jenkinsfile-docker-centos b/support/jenkins/Jenkinsfile-docker-centos
index 762462f..99575c9 100644
--- a/support/jenkins/Jenkinsfile-docker-centos
+++ b/support/jenkins/Jenkinsfile-docker-centos
@@ -9,7 +9,7 @@ node(env.NODE_LABELS) {
withCredentials([[
$class: 'UsernamePasswordMultiBinding',
- credentialsId: 'mesos-docker-bot'
+ credentialsId: 'mesos-docker-bot',
usernameVariable: 'USERNAME',
passwordVariable: 'PASSWORD']]) {
sh("support/jenkins/docker-centos.sh")
[mesos] 07/10: Added Jenkinsfile for Mesos mini.
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jieyu pushed a commit to branch 1.7.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit e59b5555955bd4eaf8c033d3c3217a9f0d57d830
Author: Jie Yu <yu...@gmail.com>
AuthorDate: Sat Oct 27 20:20:45 2018 -0700
Added Jenkinsfile for Mesos mini.
(cherry picked from commit 4e4e6b93e0df5aa19e166fd107b6e01db999e4f5)
---
support/jenkins/Jenkinsfile-mini | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/support/jenkins/Jenkinsfile-mini b/support/jenkins/Jenkinsfile-mini
new file mode 100644
index 0000000..6790d89
--- /dev/null
+++ b/support/jenkins/Jenkinsfile-mini
@@ -0,0 +1,17 @@
+#!groovy
+
+if (env.NODE_LABELS == null) {
+ env.NODE_LABELS = "(ubuntu)&&(!ubuntu-us1)&&(!ubuntu-eu2)&&(!qnode3)&&(!H23)"
+}
+
+node(env.NODE_LABELS) {
+ checkout scm
+
+ withCredentials([[
+ $class: 'UsernamePasswordMultiBinding',
+ credentialsId: 'mesos-docker-bot',
+ usernameVariable: 'USERNAME',
+ passwordVariable: 'PASSWORD']]) {
+ sh("support/jenkins/mini.sh")
+ }
+}