You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@yetus.apache.org by aw...@apache.org on 2019/01/03 07:48:24 UTC
[yetus] branch master updated: YETUS-683. Publish apache/yetus
docker images on hub.docker.com
This is an automated email from the ASF dual-hosted git repository.
aw pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/yetus.git
The following commit(s) were added to refs/heads/master by this push:
new 8c3c818 YETUS-683. Publish apache/yetus docker images on hub.docker.com
8c3c818 is described below
commit 8c3c818285ee4e03bf31c56f678924ca37cfaf6e
Author: Allen Wittenauer <aw...@apache.org>
AuthorDate: Sun Nov 11 22:26:26 2018 -0800
YETUS-683. Publish apache/yetus docker images on hub.docker.com
Signed-off-by: Allen Wittenauer <aw...@apache.org>
---
.../documentation/in-progress/precommit-basic.md | 6 +-
.../documentation/in-progress/precommit-robots.md | 59 ++++++++++++++-
asf-site-src/source/downloads.html.md.erb | 2 +-
asf-site-src/source/yetus-docker-image.md | 86 ++++++++++++++++++++++
4 files changed, 147 insertions(+), 6 deletions(-)
diff --git a/asf-site-src/source/documentation/in-progress/precommit-basic.md b/asf-site-src/source/documentation/in-progress/precommit-basic.md
index 5e24d28..63d25f2 100644
--- a/asf-site-src/source/documentation/in-progress/precommit-basic.md
+++ b/asf-site-src/source/documentation/in-progress/precommit-basic.md
@@ -84,7 +84,7 @@ Build Tools:
Automation and Isolation:
* [Circle CI](https://www.circleci.com)
-* [Docker](https://www.docker.com) version 1.6.0+
+* [Docker](https://www.docker.com) version 1.7.0+
* [Gitlab CI](https://www.gitlab.com)
* [Jenkins](https://www.jenkins-ci.org)
* [Travis CI](https://www.travis-ci.com)
@@ -329,10 +329,10 @@ NOTE: JAVA\_HOME is always appended to the list of JDKs in MultiJDK mode. If JA
`test-patch` also has a mode to utilize Docker:
```bash
-$ test-patch. (other options) --docker
+$ test-patch (other options) --docker
```
-This will do some preliminary setup and then re-execute itself inside a Docker container. For more information on how to provide a custom Dockerfile and other Docker-specific features, see [precommit-docker](../precommit-docker) for more information.
+This command will do some preliminary setup and then re-execute itself inside a Docker container. For more information on how to provide a custom Dockerfile and other Docker-specific features, see the specific [precommit Docker support](../precommit-docker) page and the [Apache Yetus Docker Hub Images](/yetus-docker-image) page for more information on the convenience Docker images.
# In Closing
diff --git a/asf-site-src/source/documentation/in-progress/precommit-robots.md b/asf-site-src/source/documentation/in-progress/precommit-robots.md
index b0fab18..faa487c 100644
--- a/asf-site-src/source/documentation/in-progress/precommit-robots.md
+++ b/asf-site-src/source/documentation/in-progress/precommit-robots.md
@@ -38,6 +38,23 @@ TRIGGER: ${CIRCLECI}=true
Circle CI support in `test-patch` is limited to github.com. Artifacts (the `--patch-dir` directory) location needs to be handled set on the command line. Linking to the logs is not currently supported.
+To use the pre-built Apache Yetus Docker image from docker hub as the build environment, use the following snippet in the `.circleci/config.yaml` file, substituting the tag for the version of Apache Yetus that should be used and replacing the JAVA_HOME with the appropriate version as bundled mentioned in the Dockerfile:
+
+```yaml
+jobs:
+ build:
+ docker:
+ - image: apache/yetus:0.9.0
+
+ environment:
+ JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+
+ ...
+```
+
+See also
+ * See also the source tree's `.circleci/config.yaml` for some tips and tricks.
+
Gitlab CI
=========
@@ -54,6 +71,21 @@ Artifacts, patch logs, etc are configured to go to a yetus-out directory in the
```
+To use the pre-built Apache Yetus Docker image from docker hub as the build environment, use the following snippet in the `.gitlab-ci.yml` file, substituting the tag for the version of Apache Yetus that should be used and replacing the JAVA_HOME with the appropriate version as bundled mentioned in the Dockerfile:
+
+```yaml
+job:
+ image: apache/yetus:0.9.0
+ allow_failure: true
+ variables:
+ JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+
+ ...
+```
+
+See also
+ * See also the source tree's `.gitlab-ci.yml` for some tips and tricks.
+
Jenkins
=======
@@ -66,9 +98,29 @@ There is some support for a few well known environment variables:
* `${GIT_URL}` will trigger the same extra handling if 'github' or 'gitlab' appear in the string.
* If `${ghprbPullId}` is set, then test-patch will configure itself for a Github-style PR.
+To use the pre-built Apache Yetus Docker image from docker hub as the build environment, use the following snippet in the `Jenkinsfile`, substituting the tag for the version of Apache Yetus that should be used and replacing the JAVA_HOME with the appropriate version as bundled mentioned in the Dockerfile:
+
+```groovy
+pipeline {
+ agent {
+ docker {
+ image 'apache/yetus:0.9.0'
+ args '-v /var/run/docker.sock:/var/run/docker.sock'
+ }
+ }
+
+ environment {
+ JAVA_HOME = '/usr/lib/jvm/java-8-openjdk-amd64'
+ }
+
+ ...
+
+}
+
+```
See also
- * See also the source tree's Jenkinsfile for some tips and tricks.
+ * See also the source tree's `Jenkinsfile` for some tips and tricks.
* [precommit-admin](precommit-admin), for special utilities built for Jenkins.
* [GitHub Branch Source Plugin](https://wiki.jenkins.io/display/JENKINS/GitHub+Branch+Source+Plugin)
* [GitHub Pull Request Builder Plugin](https://wiki.jenkins.io/display/JENKINS/GitHub+pull+request+builder+plugin)
@@ -83,9 +135,12 @@ Travis CI support will update the local checked out source repository to include
If `${ARTIFACTS_PATH}` is configured, then `--patch-dir` is set to the first listed directory path. However, links to the location logs must still be configured manually.
-Personalities will override the auto-detected Github repository information. It may be necessary to manually configure it in your .travis.yml file.
+Personalities will override the auto-detected Github repository information. It may be necessary to manually configure it in your `.travis.yml` file.
+As of this writing, it is not possible to make the Travis CI build environment use the Apache Yetus pre-built docker images without using ` docker run` in the before_install phase. Therefore, using the image is the same as described in the [Apache Yetus Docker Hub Images](/yetus-docker-image) page.
+See also
+ * See also the source tree's `.travis.yml` for some tips and tricks.
Manual Configuration
====================
diff --git a/asf-site-src/source/downloads.html.md.erb b/asf-site-src/source/downloads.html.md.erb
index e3c0f11..27c362e 100644
--- a/asf-site-src/source/downloads.html.md.erb
+++ b/asf-site-src/source/downloads.html.md.erb
@@ -31,7 +31,7 @@ should verify the integrity of these artifacts via [the ASF procedure](https://w
using the [KEYS file for our project](https://dist.apache.org/repos/dist/release/yetus/KEYS).
In addition to the artifacts below, we publish convenience artifacts in some
-third-party repositories (e.g. Maven Central).
+third-party repositories (e.g. Maven Central) and [Docker Hub](../yetus-docker-image).
## Releases
diff --git a/asf-site-src/source/yetus-docker-image.md b/asf-site-src/source/yetus-docker-image.md
new file mode 100644
index 0000000..963cbf5
--- /dev/null
+++ b/asf-site-src/source/yetus-docker-image.md
@@ -0,0 +1,86 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+Convenience Docker Hub Images
+===========================
+
+While not official releases, the Apache Yetus project makes available two types of images on hub.docker.com:
+
+ * apache/yetus-base
+
+ This image is the same as the 'built-in' Dockerfile when `--docker` is used without `--dockerfile` on the precommit command line. It includes all of the pre-requisites as needed by the various Apache Yetus components. It is located in `precommit/src/main/shell/test-patch-docker/`.
+
+ * apache/yetus
+
+ This image is the same as apache/yetus-base but includes a pre-built version of Apache Yetus as part of the base OS image. In other words, qbt, releasedocmaker, shelldocs, test-patch, etc., are in /usr/bin and available in the default path. It is generated from the Dockerfile located in the root of the source and is built with the options provided in the hooks directory.
+
+Both images should be suitable to be used as a building block or even directly if your build environment needs no other dependencies. These images are especially useful for various CI systems that require a Docker image to be used.
+
+Images are tagged such that 'master' represents the last successful Docker image build of the master branch. Images based off of the official source releases are tagged with a matching version number (e.g., 0.9.0). There is no 'latest' tagged image. It is recommended that users choose a stable tag so as not to be surprised by incompatible changes.
+
+File Access
+========
+
+All of the executables that Apache Yetus provides requires access to one or more directories. These directories should be provided on the command line via the volume flags to docker run. For example, to run shelldocs against test-patch.sh:
+
+```bash
+docker run \
+ --rm \
+ -v /tmp/out:/output \
+ -v /src/precommit/src/main/shell:/input:ro \
+ apache/yetus:0.9.0 \
+ shelldocs --output /output/test-patch.md --input /input/test-patch.sh
+```
+
+In this example, we mount two volumes: one for input (marked read-only), one for output. After execution, shelldocs has processed test-patch.sh and generated a test-patch.md file in /tmp/out on our local machine.
+
+Precommit also works, assuming that Apache Yetus image has all of your project's dependencies. For example:
+
+```bash
+docker run \
+ --rm \
+ -v /tmp/patchdir:/patchdir \
+ -v /src:/basedir \
+ apache/yetus:0.9.0 \
+ qbt \
+ --plugins=all \
+ --basedir=/basedir \
+ --patch-dir=/patchdir \
+ --project=yetus
+```
+If your project needs additional dependencies, it is trivial to build off of the Apache Yetus image:
+
+```Dockerfile
+FROM apache/yetus:0.9.0
+RUN apt-get -q update && apt-get -q install -y \
+ clang \
+ libssl-dev \
+ valgrind \
+ zlib1g-dev
+```
+ ```bash
+docker build -t project/build:0.9.0 -f .
+```
+
+This example builds a docker image based off of Apache Yetus 0.9.0 but with the additions of clang, some development libraries, and valgrind. Now project/build:0.9.0 can be used instead of apache/yetus:0.9.0 since it has all of Apache Yetus and the additions needed by our project.
+
+A Note About Precommit
+================
+
+test-patch and friends have direct support for Docker outside of the convenience images. That information is covered in-depth in the[precommit-docker](../precommit-docker) section.