You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2019/09/17 19:07:12 UTC
[beam] branch asf-site updated: Publishing website 2019/09/17
19:06:59 at commit 6d56040
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 497c18c Publishing website 2019/09/17 19:06:59 at commit 6d56040
497c18c is described below
commit 497c18ce7f7aa06d5c45ff8c9499180a6bd17f88
Author: jenkins <bu...@apache.org>
AuthorDate: Tue Sep 17 19:06:59 2019 +0000
Publishing website 2019/09/17 19:06:59 at commit 6d56040
---
.../contribute/release-guide/index.html | 111 ++++++++++++++++++++-
1 file changed, 110 insertions(+), 1 deletion(-)
diff --git a/website/generated-content/contribute/release-guide/index.html b/website/generated-content/contribute/release-guide/index.html
index 803566f..9385dad 100644
--- a/website/generated-content/contribute/release-guide/index.html
+++ b/website/generated-content/contribute/release-guide/index.html
@@ -271,6 +271,7 @@
<ul>
<li><a href="#deploy-artifacts-to-maven-central-repository">Deploy artifacts to Maven Central Repository</a></li>
<li><a href="#deploy-python-artifacts-to-pypi">Deploy Python artifacts to PyPI</a></li>
+ <li><a href="#deploy-sdk-docker-images-to-dockerhub">Deploy SDK docker images to DockerHub</a></li>
<li><a href="#git-tag">Git tag</a></li>
<li><a href="#merge-website-pull-request">Merge website pull request</a></li>
<li><a href="#mark-the-version-as-released-in-jira">Mark the version as released in JIRA</a></li>
@@ -327,6 +328,7 @@ limitations under the License.
<li><a href="#submit-your-gpg-public-key-into-mit-pgp-public-key-server" id="markdown-toc-submit-your-gpg-public-key-into-mit-pgp-public-key-server">Submit your GPG public key into MIT PGP Public Key Server</a></li>
<li><a href="#website-development-setup" id="markdown-toc-website-development-setup">Website development setup</a></li>
<li><a href="#register-to-pypi" id="markdown-toc-register-to-pypi">Register to PyPI</a></li>
+ <li><a href="#login-to-dockerhub" id="markdown-toc-login-to-dockerhub">Login to DockerHub</a></li>
</ul>
</li>
<li><a href="#create-a-new-version-in-jira" id="markdown-toc-create-a-new-version-in-jira">Create a new version in JIRA</a></li>
@@ -364,6 +366,7 @@ limitations under the License.
<li><a href="#build-and-stage-java-artifacts-with-gradle" id="markdown-toc-build-and-stage-java-artifacts-with-gradle">Build and stage Java artifacts with Gradle</a></li>
<li><a href="#stage-source-release-on-distapacheorg" id="markdown-toc-stage-source-release-on-distapacheorg">Stage source release on dist.apache.org</a></li>
<li><a href="#stage-python-binaries-on-distapacheorg" id="markdown-toc-stage-python-binaries-on-distapacheorg">Stage python binaries on dist.apache.org</a></li>
+ <li><a href="#stage-sdk-images-on-hubdockercom" id="markdown-toc-stage-sdk-images-on-hubdockercom">Stage SDK images on hub.docker.com</a></li>
</ul>
</li>
<li><a href="#build-and-stage-python-wheels" id="markdown-toc-build-and-stage-python-wheels">Build and stage python wheels</a></li>
@@ -395,6 +398,7 @@ limitations under the License.
<li><a href="#deploy-source-release-to-distapacheorg" id="markdown-toc-deploy-source-release-to-distapacheorg">Deploy source release to dist.apache.org</a></li>
</ul>
</li>
+ <li><a href="#deploy-sdk-docker-images-to-dockerhub" id="markdown-toc-deploy-sdk-docker-images-to-dockerhub">Deploy SDK docker images to DockerHub</a></li>
<li><a href="#git-tag" id="markdown-toc-git-tag">Git tag</a></li>
<li><a href="#merge-website-pull-request" id="markdown-toc-merge-website-pull-request">Merge website pull request</a></li>
<li><a href="#mark-the-version-as-released-in-jira" id="markdown-toc-mark-the-version-as-released-in-jira">Mark the version as released in JIRA</a></li>
@@ -472,6 +476,7 @@ please follow <a href="https://help.github.com/articles/creating-a-personal-acce
<li>GPG pass phrase (see the next section);</li>
<li>Apache ID and Password;</li>
<li>GitHub ID and Password.</li>
+ <li>DockerHub ID and Password. (You should be a member of maintainer team; email at dev@ if you are not.)</li>
</ul>
<h3 id="one-time-setup-instructions">One-time setup instructions</h3>
@@ -633,6 +638,32 @@ Automation Reliability</a>.</p>
<p>Release manager needs to have an account with PyPI. If you need one, <a href="https://pypi.python.org/account/register/">register at PyPI</a>. You also need to be a maintainer (or an owner) of the <a href="https://pypi.python.org/pypi/apache-beam">apache-beam</a> package in order to push a new release. Ask on the mailing list for assistance.</p>
+<h4 id="login-to-dockerhub">Login to DockerHub</h4>
+<p>Run following command manually. It will ask you to input your DockerHub ID and password if
+authorization info cannot be found from ~/.docker/config.json file.</p>
+<div class="highlighter-rouge"><pre class="highlight"><code>docker login docker.io
+</code></pre>
+</div>
+<p>After successful login, authorization info will be stored at ~/.docker/config.json file. For example,</p>
+<div class="highlighter-rouge"><pre class="highlight"><code>"https://index.docker.io/v1/": {
+ "auth": "aGFubmFoamlhbmc6cmtkdGpmZ2hrMTIxMw=="
+}
+</code></pre>
+</div>
+<p>Release managers should have push permission; please ask for help at dev@.</p>
+<div class="highlighter-rouge"><pre class="highlight"><code>From: Release Manager
+To: dev@beam.apache.org
+Subject: DockerHub Push Permission
+
+Hi DockerHub Admins
+
+I need push permission to proceed with release, can you please add me to maintainer team?
+My docker hub ID is: xxx
+
+Thanks,
+Release Manager
+</code></pre>
+</div>
<h3 id="create-a-new-version-in-jira">Create a new version in JIRA</h3>
<p>When contributors resolve an issue in JIRA, they are tagging it with a release that will contain their changes. With the release currently underway, new issues should be resolved against a subsequent future release. Therefore, you should create a release item for this subsequent release, as follows:</p>
@@ -685,7 +716,8 @@ Automation Reliability</a>.</p>
<p>Change and commit version number in release branch:</p>
<p><a href="https://github.com/apache/beam/blob/release-2.6.0/sdks/python/apache_beam/version.py#L21">version.py</a>,
-<a href="https://github.com/apache/beam/blob/release-2.6.0/runners/google-cloud-dataflow-java/build.gradle#L39">build.gradle</a></p>
+<a href="https://github.com/apache/beam/blob/release-2.6.0/runners/google-cloud-dataflow-java/build.gradle#L39">build.gradle</a>,
+<a href="https://github.com/apache/beam/blob/release-2.16.0/gradle.properties#L27">gradle.properties</a></p>
</li>
</ol>
</li>
@@ -759,6 +791,7 @@ git commit -m "Moving to ${NEXT_VERSION_IN_BASE_BRANCH}-SNAPSHOT on master branc
<div class="highlighter-rouge"><pre class="highlight"><code>DEV=${RELEASE}.dev
sed -i -e "s/${DEV}/${RELEASE}/g" sdks/python/apache_beam/version.py
+sed -i -e "s/${DEV}/${RELEASE}/g" gradle.properties
sed -i -e "s/'beam-master-.*'/'beam-${RELEASE}'/g" runners/google-cloud-dataflow-java/build.gradle
</code></pre>
</div>
@@ -1063,6 +1096,7 @@ When prompted for a description, enter “Apache Beam, version X, release candid
</li>
<li>Stage source release into dist.apache.org dev <a href="https://dist.apache.org/repos/dist/dev/beam/">repo</a>.</li>
<li>Stage,sign and hash python binaries into dist.apache.ord dev repo python dir</li>
+ <li>Stage SDK docker images to <a href="https://hub.docker.com/u/apachebeam">https://hub.docker.com/u/apachebeam</a>.</li>
<li>Create a PR to update beam and beam-site, changes includes:
<ul>
<li>Copy python doc into beam-site</li>
@@ -1195,6 +1229,53 @@ svn commit
<p>Verify that files are <a href="https://dist.apache.org/repos/dist/dev/beam">present</a>.</p>
+<h4 id="stage-sdk-images-on-hubdockercom">Stage SDK images on hub.docker.com</h4>
+<ul>
+ <li>Build Python images and push to DockerHub.</li>
+</ul>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>./gradlew :sdks:python:container:buildAll -Pdocker-tag=${RELEASE}_rc{RC_NUM}
+
+PYTHON_VER=("python2.7" "python3.5" "python3.6" "python3.7")
+for ver in "${PYTHON_VER[@]}"; do
+ docker push apachebeam/${ver}_sdk:${RELEASE}_rc{RC_NUM} &
+done
+</code></pre>
+</div>
+
+<ul>
+ <li>Build Java images and push to DockerHub.</li>
+</ul>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>./gradlew :sdks:java:container:dockerPush -Pdocker-tag=${RELEASE}_rc{RC_NUM}
+</code></pre>
+</div>
+
+<ul>
+ <li>Build Go images and push to DockerHub.</li>
+</ul>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>./gradlew :sdks:go:container:dockerPush -Pdocker-tag=${RELEASE}_rc{RC_NUM}
+</code></pre>
+</div>
+
+<p>Clean up images from local</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>for ver in "${PYTHON_VER[@]}"; do
+ docker rmi -f apachebeam/${ver}_sdk:${RELEASE}_rc{RC_NUM}
+done
+docker rmi -f apachebeam/java_sdk:${RELEASE}_rc{RC_NUM}
+docker rmi -f apachebeam/go_sdk:${RELEASE}_rc{RC_NUM}
+</code></pre>
+</div>
+
+<p>How to find images:</p>
+<ol>
+ <li>Visit <a href="https://hub.docker.com/u/apachebeam">https://hub.docker.com/u/apachebeam</a></li>
+ <li>Visit each repository and navigate to <em>tags</em> tab.</li>
+ <li>Verify images are pushed with tags: ${RELEASE}_rc{RC_NUM}</li>
+</ol>
+
<h3 id="build-and-stage-python-wheels">Build and stage python wheels</h3>
<p>There is a wrapper repo <a href="https://github.com/apache/beam-wheels">beam-wheels</a> to help build python wheels.</p>
@@ -1358,6 +1439,7 @@ all major features and bug fixes, and all known issues.</p>
<li>Maven artifacts deployed to the staging repository of <a href="https://repository.apache.org/content/repositories/">repository.apache.org</a></li>
<li>Source distribution deployed to the dev repository of <a href="https://dist.apache.org/repos/dist/dev/beam/">dist.apache.org</a></li>
<li>Website pull request proposed to list the <a href="/get-started/downloads/">release</a>, publish the <a href="https://beam.apache.org/releases/javadoc/">Java API reference manual</a>, and publish the <a href="https://beam.apache.org/releases/pydoc/">Python API reference manual</a>.</li>
+ <li>Docker images are published to <a href="https://hub.docker.com/u/apachebeam">DockerHub</a> with tags: {RELEASE}_rc{RC_NUM}.</li>
</ol>
<p>You can (optionally) also do additional verification by:</p>
@@ -1367,6 +1449,12 @@ all major features and bug fixes, and all known issues.</p>
<li>Check signatures (e.g. <code class="highlighter-rouge">gpg --verify apache-beam-1.2.3-python.zip.asc apache-beam-1.2.3-python.zip</code>)</li>
<li><code class="highlighter-rouge">grep</code> for legal headers in each file.</li>
<li>Run all jenkins suites and include links to passing tests in the voting email. (Select “Run with parameters”)</li>
+ <li>Pull docker images to make sure they are pullable.
+ <div class="highlighter-rouge"><pre class="highlight"><code>docker pull {image_name}
+docker pull apachebeam/python3.5_sdk:2.16.0_rc1
+</code></pre>
+ </div>
+ </li>
</ol>
<hr />
@@ -1396,6 +1484,7 @@ The complete staging area is available for your review, which includes:
* Java artifacts were built with Maven MAVEN_VERSION and OpenJDK/Oracle JDK JDK_VERSION.
* Python artifacts are deployed along with the source release to the dist.apache.org [2].
* Validation sheet with a tab for 1.2.3 release to help with validation [9].
+* Docker images puhlished to Docker Hub [10].
The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes.
@@ -1411,6 +1500,7 @@ Release Manager
[7] https://github.com/apache/beam-site/pull/...
[8] https://github.com/apache/beam/pull/...
[9] https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=...
+[10] https://hub.docker.com/u/apachebeam
</code></pre>
</div>
@@ -1789,6 +1879,25 @@ delete the <code class="highlighter-rouge">.asc</code>, <code class="highlighter
<p>Make sure the download address for last release version is upldaed, <a href="https://github.com/apache/beam-site/pull/478">example PR</a>.</p>
+<h3 id="deploy-sdk-docker-images-to-dockerhub">Deploy SDK docker images to DockerHub</h3>
+<p>TODO(hannahjiang): change link to master branch after #9560 is merged.</p>
+
+<ul>
+ <li>Script: <a href="https://github.com/Hannah-Jiang/beam/blob/release_script_for_containers/release/src/main/scripts/publish_docker_images.sh">publish_docker_images.sh</a></li>
+ <li>Usage
+ <div class="highlighter-rouge"><pre class="highlight"><code>./beam/release/src/main/scripts/publish_docker_images.sh
+</code></pre>
+ </div>
+ <p>Verify that:</p>
+ </li>
+ <li>Images are published at <a href="https://hub.docker.com/u/apachebeam">DockerHub</a> with tags {RELEASE} and <em>latest</em>.</li>
+ <li>Images with <em>latest</em> tag are pointing to current release by confirming
+ <ol>
+ <li>Digest of the image with <em>latest</em> tag is the same as the one with {RELEASE} tag.</li>
+ </ol>
+ </li>
+</ul>
+
<h3 id="git-tag">Git tag</h3>
<p>Create and push a new signed tag for the released version by copying the tag for the final release candidate, as follows:</p>