You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by vi...@apache.org on 2020/09/11 14:26:13 UTC
[mesos] branch master updated (5d68474 -> f86cb65)
This is an automated email from the ASF dual-hosted git repository.
vinodkone pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git.
from 5d68474 Added MESOS-10134 to the 1.7.4 CHANGELOG.
new a5b7765 Fixed Website bot to work with docker user namespaces.
new af01d50 Merged build.sh and entrypoint.sh.
new 0a9e90a Reduced build parallelism to avoid aborted builds in CI.
new 28cabfc Added `--no-same-owner` option to tar command.
new 311c47e Added `--no-same-owner` option to tar command.
new d335288 Added `--no-same-owner` option to tar command.
new f86cb65 Added cleanup of build dir to website bot.
The 7 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
3rdparty/Makefile.am | 4 +--
3rdparty/libprocess/3rdparty/Makefile.am | 2 +-
3rdparty/stout/3rdparty/Makefile.am | 2 +-
support/mesos-website.sh | 10 +++----
support/mesos-website/Dockerfile | 6 +++--
support/mesos-website/build.sh | 44 -------------------------------
support/mesos-website/entrypoint.sh | 45 +++++++++++++++++++++++++-------
7 files changed, 47 insertions(+), 66 deletions(-)
delete mode 100755 support/mesos-website/build.sh
[mesos] 01/07: Fixed Website bot to work with docker user
namespaces.
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
vinodkone pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit a5b7765d3ef2905d11ffadbb877377b07624a99c
Author: Vinod Kone <vi...@gmail.com>
AuthorDate: Wed Sep 9 17:20:58 2020 -0500
Fixed Website bot to work with docker user namespaces.
ASF CI has enabled user namespaces for docker daemon on the build
machines that are used to build the website. This commit makes the
requisite changes to work with user namespacing.
Review: https://reviews.apache.org/r/72856
---
support/mesos-website.sh | 10 +++-------
support/mesos-website/Dockerfile | 6 ++++--
support/mesos-website/build.sh | 13 +++++++++++++
support/mesos-website/entrypoint.sh | 12 ++++--------
4 files changed, 24 insertions(+), 17 deletions(-)
diff --git a/support/mesos-website.sh b/support/mesos-website.sh
index 04a6646..af98f36 100755
--- a/support/mesos-website.sh
+++ b/support/mesos-website.sh
@@ -32,15 +32,11 @@ docker build --no-cache=true -t $TAG support/mesos-website
trap 'docker rmi $TAG' EXIT
-# NOTE: We set `LOCAL_USER_ID` environment variable to enable running the
-# container process with the same UID as the user running this script. This
-# ensures that any writes to the mounted volumes will have the same permissions
-# as the user running the script, making it easy to do cleanups; otherwise
-# any writes will have permissions of UID 0 by default on Linux.
-
+# NOTE: ASF CI remaps the host `jenkins` user to UID 0 inside the container
+# so we don't need to do any user switching inside the container to be able
+# to write to the mounted host volumes with the correct permissions.
docker run \
--rm \
- -e LOCAL_USER_ID="$(id -u "$USER")" \
-v "$MESOS_DIR":/mesos:Z \
-v "$MESOS_SITE_DIR/content":/mesos/site/publish:Z \
$TAG
diff --git a/support/mesos-website/Dockerfile b/support/mesos-website/Dockerfile
index 611c496..bcd62fe 100644
--- a/support/mesos-website/Dockerfile
+++ b/support/mesos-website/Dockerfile
@@ -3,8 +3,10 @@ MAINTAINER "dev@mesos.apache.org"
LABEL Description="This image is used for generating Mesos web site."
-# The mesos build image drops down to user `mesos`, but
-# we need priviledged access to install packages below.
+# Set the root user explicitly because the base image
+# `mesos/mesos-build` uses `mesos` user by default. Also, note that
+# `root` inside the container is mapped to `jenkins` user on the build
+# machine host in ASF CI.
USER root
# Install dependencies.
diff --git a/support/mesos-website/build.sh b/support/mesos-website/build.sh
index afbec48..45ab892 100755
--- a/support/mesos-website/build.sh
+++ b/support/mesos-website/build.sh
@@ -27,6 +27,19 @@ function exit_hook {
trap exit_hook EXIT
+file_owner_uid=`stat . --format=%u`
+current_user_uid=`id -u`
+if [ $file_owner_uid -ne $current_user_uid ];
+then
+ echo "
+ The mounted mesos sources are owned by UID $file_owner_uid
+ which is different from the current user UID $current_user_uid
+ inside the container. Please check that dockerd has
+ user namespace remapping configured properly.
+ "
+ exit 1
+fi
+
# Build mesos to get the latest master and agent binaries.
./bootstrap
mkdir -p build
diff --git a/support/mesos-website/entrypoint.sh b/support/mesos-website/entrypoint.sh
index 72fd723..2029949 100755
--- a/support/mesos-website/entrypoint.sh
+++ b/support/mesos-website/entrypoint.sh
@@ -16,18 +16,14 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-# This is a wrapper script for building Mesos website as a non-root user.
+# This is a wrapper script for building Mesos website.
+# TODO(vinod): Merge this with `build.sh` now that we dont need to switch
+# users in ASF CI as user namespacing has been implemented.
set -e
set -o pipefail
-# This needs to be run under `root` user for `bundle exec rake` to
-# work properly. See MESOS-7859.
pushd site
bundle install
popd # site
-# Create a local user account.
-useradd -u $LOCAL_USER_ID -s /bin/bash -m tempuser
-
-# Build mesos and the website as the new user.
-su -c support/mesos-website/build.sh tempuser
+./support/mesos-website/build.sh
[mesos] 03/07: Reduced build parallelism to avoid aborted builds in
CI.
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
vinodkone pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit 0a9e90af54edc4a1c48a406b54404246a86d1499
Author: Vinod Kone <vi...@gmail.com>
AuthorDate: Thu Sep 10 07:09:14 2020 -0500
Reduced build parallelism to avoid aborted builds in CI.
Review: https://reviews.apache.org/r/72858
---
support/mesos-website/entrypoint.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/support/mesos-website/entrypoint.sh b/support/mesos-website/entrypoint.sh
index 6ecfbe0..2cc4ab2 100755
--- a/support/mesos-website/entrypoint.sh
+++ b/support/mesos-website/entrypoint.sh
@@ -44,7 +44,7 @@ fi
mkdir -p build
pushd build
../configure --disable-python
-make -j6
+make -j3 # Higher parallelism sometimes causes CI to get stuck.
popd # build
# Generate the endpoint docs from the latest mesos and agent binaries.
[mesos] 07/07: Added cleanup of build dir to website bot.
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
vinodkone pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit f86cb659fc38a198261b8d62bd463f8d3eb313f0
Author: Vinod Kone <vi...@gmail.com>
AuthorDate: Thu Sep 10 16:30:12 2020 -0500
Added cleanup of build dir to website bot.
Review: https://reviews.apache.org/r/72862
---
support/mesos-website/entrypoint.sh | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/support/mesos-website/entrypoint.sh b/support/mesos-website/entrypoint.sh
index 2cc4ab2..c8af020 100755
--- a/support/mesos-website/entrypoint.sh
+++ b/support/mesos-website/entrypoint.sh
@@ -21,7 +21,10 @@ set -e
set -o pipefail
function exit_hook {
- # Remove generated documents when exit.
+ # Remove mesos build directory on exit.
+ rm -rf /mesos/build
+
+ # Remove generated documents on exit.
cd /mesos/site && bundle exec rake clean_docs
}
[mesos] 04/07: Added `--no-same-owner` option to tar command.
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
vinodkone pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit 28cabfc61248908c42c91d4287730cad42dd0400
Author: Vinod Kone <vi...@gmail.com>
AuthorDate: Thu Sep 10 15:18:44 2020 -0500
Added `--no-same-owner` option to tar command.
This ensures extracted tarballs are owned by root when running the
build as root.
Review: https://reviews.apache.org/r/72859
---
3rdparty/stout/3rdparty/Makefile.am | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/3rdparty/stout/3rdparty/Makefile.am b/3rdparty/stout/3rdparty/Makefile.am
index 5a88889..7edb7d2 100644
--- a/3rdparty/stout/3rdparty/Makefile.am
+++ b/3rdparty/stout/3rdparty/Makefile.am
@@ -71,7 +71,7 @@ CLEAN_EXTRACTED = \
# which cause the packages to get extracted as necessary. We also
# apply any patches as appropriate.
%-stamp: $(BUNDLED_DIR)/%.tar.gz
- gzip -d -c $^ | tar xf -
+ gzip -d -c $^ | tar xf - --no-same-owner
test ! -e $(top_srcdir)/../$*.patch || patch -d $* -p1 <$(top_srcdir)/../$*.patch
touch $@
[mesos] 02/07: Merged build.sh and entrypoint.sh.
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
vinodkone pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit af01d50663e3c1ec81177dd8baeecae3e5ba8819
Author: Vinod Kone <vi...@gmail.com>
AuthorDate: Wed Sep 9 20:17:50 2020 -0500
Merged build.sh and entrypoint.sh.
Review: https://reviews.apache.org/r/72857
---
support/mesos-website/build.sh | 57 -------------------------------------
support/mesos-website/entrypoint.sh | 38 +++++++++++++++++++++----
2 files changed, 33 insertions(+), 62 deletions(-)
diff --git a/support/mesos-website/build.sh b/support/mesos-website/build.sh
deleted file mode 100755
index 45ab892..0000000
--- a/support/mesos-website/build.sh
+++ /dev/null
@@ -1,57 +0,0 @@
-#!/usr/bin/env bash
-
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# This is a script for building Mesos website.
-set -e
-set -o pipefail
-
-function exit_hook {
- # Remove generated documents when exit.
- cd /mesos/site && bundle exec rake clean_docs
-}
-
-trap exit_hook EXIT
-
-file_owner_uid=`stat . --format=%u`
-current_user_uid=`id -u`
-if [ $file_owner_uid -ne $current_user_uid ];
-then
- echo "
- The mounted mesos sources are owned by UID $file_owner_uid
- which is different from the current user UID $current_user_uid
- inside the container. Please check that dockerd has
- user namespace remapping configured properly.
- "
- exit 1
-fi
-
-# Build mesos to get the latest master and agent binaries.
-./bootstrap
-mkdir -p build
-pushd build
-../configure --disable-python
-make -j6
-popd # build
-
-# Generate the endpoint docs from the latest mesos and agent binaries.
-./support/generate-endpoint-help.py
-
-# Build the website.
-pushd site
-bundle exec rake
-popd # site
diff --git a/support/mesos-website/entrypoint.sh b/support/mesos-website/entrypoint.sh
index 2029949..6ecfbe0 100755
--- a/support/mesos-website/entrypoint.sh
+++ b/support/mesos-website/entrypoint.sh
@@ -16,14 +16,42 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-# This is a wrapper script for building Mesos website.
-# TODO(vinod): Merge this with `build.sh` now that we dont need to switch
-# users in ASF CI as user namespacing has been implemented.
+# This is a script for building Mesos website.
set -e
set -o pipefail
+function exit_hook {
+ # Remove generated documents when exit.
+ cd /mesos/site && bundle exec rake clean_docs
+}
+
+trap exit_hook EXIT
+
+file_owner_uid=`stat . --format=%u`
+current_user_uid=`id -u`
+if [ $file_owner_uid -ne $current_user_uid ]; then
+ echo "
+ The mounted mesos sources are owned by UID $file_owner_uid
+ which is different from the current user UID $current_user_uid
+ inside the container. Please check that dockerd has
+ user namespace remapping configured properly.
+ "
+ exit 1
+fi
+
+# Build mesos to get the latest master and agent binaries.
+./bootstrap
+mkdir -p build
+pushd build
+../configure --disable-python
+make -j6
+popd # build
+
+# Generate the endpoint docs from the latest mesos and agent binaries.
+./support/generate-endpoint-help.py
+
+# Build the website.
pushd site
bundle install
+bundle exec rake
popd # site
-
-./support/mesos-website/build.sh
[mesos] 05/07: Added `--no-same-owner` option to tar command.
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
vinodkone pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit 311c47e4a9169bdc6cdc962383c711a38c7167d2
Author: Vinod Kone <vi...@gmail.com>
AuthorDate: Thu Sep 10 15:20:41 2020 -0500
Added `--no-same-owner` option to tar command.
This ensures extracted tarballs are owned by root when running the
build as root.
Review: https://reviews.apache.org/r/72860
---
3rdparty/libprocess/3rdparty/Makefile.am | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/3rdparty/libprocess/3rdparty/Makefile.am b/3rdparty/libprocess/3rdparty/Makefile.am
index e148f2d..77e9073 100644
--- a/3rdparty/libprocess/3rdparty/Makefile.am
+++ b/3rdparty/libprocess/3rdparty/Makefile.am
@@ -97,7 +97,7 @@ CLEAN_EXTRACTED = \
# which cause the packages to get extracted as necessary. We also
# apply any patches as appropriate.
%-stamp: $(BUNDLED_DIR)/%.tar.gz
- gzip -d -c $^ | tar xf -
+ gzip -d -c $^ | tar xf - --no-same-owner
test ! -e $(top_srcdir)/../$*.patch || patch -d $* -p1 <$(top_srcdir)/../$*.patch
touch $@
[mesos] 06/07: Added `--no-same-owner` option to tar command.
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
vinodkone pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit d3352889d075bb84ed86c69e8acd9b52697d0a1b
Author: Vinod Kone <vi...@gmail.com>
AuthorDate: Thu Sep 10 15:21:37 2020 -0500
Added `--no-same-owner` option to tar command.
This ensures extracted tarballs are owned by root when running the
build as root.
Review: https://reviews.apache.org/r/72861
---
3rdparty/Makefile.am | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/3rdparty/Makefile.am b/3rdparty/Makefile.am
index 23f49ec..26099e2 100644
--- a/3rdparty/Makefile.am
+++ b/3rdparty/Makefile.am
@@ -178,7 +178,7 @@ CLEAN_EXTRACTED = \
# which cause the packages to get extracted as necessary. We also
# apply any patches as appropriate.
%-stamp: %.tar.gz
- gzip -d -c $^ | tar xf -
+ gzip -d -c $^ | tar xf - --no-same-owner
test ! -e $(srcdir)/$*.patch || patch -d $* -p1 <$(srcdir)/$*.patch
touch $@
@@ -191,7 +191,7 @@ CLEAN_EXTRACTED = \
CSI_STAMPS = $(CSI_V0)-stamp $(CSI_V1)-stamp
$(CSI_STAMPS): csi-%-stamp: csi-%.tar.gz
$(MKDIR_P) csi-$*
- gzip -d -c $^ | (cd csi-$* && tar xf -)
+ gzip -d -c $^ | (cd csi-$* && tar xf - --no-same-owner)
test ! -e $(srcdir)/csi-$*.patch || \
patch -d csi-$*/spec-$* -p1 <$(srcdir)/csi-$*.patch
touch $@