You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by me...@apache.org on 2018/08/14 19:56:01 UTC
[beam-site] branch mergebot updated (b76e7b4 -> 0b58334)
This is an automated email from the ASF dual-hosted git repository.
mergebot-role pushed a change to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git.
discard b76e7b4 This closes #536
discard 61291b7 Grammar pass + pretty print
discard b29ac7c Address readability reviews
discard cfe7ebb Quick fixes to layout
discard d73b18d Fixing 2.6.0 blog post
new 33288e9 Add instructions of using automation scripts
new 115d974 Fix broken tests
new 77d3348 Addressed Ahmet's comments
new 0b58334 This closes #531
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (b76e7b4)
\
N -- N -- N refs/heads/mergebot (0b58334)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
src/_data/authors.yml | 2 +-
src/_includes/authors-list.md | 13 +
src/_posts/2018-08-10-beam-2.6.0.md | 75 -----
src/_posts/src/_posts/2018-08-10-beam-2.6.0.md | 58 ++++
src/contribute/release-guide.md | 383 +++++++++++++++++--------
5 files changed, 338 insertions(+), 193 deletions(-)
delete mode 100644 src/_posts/2018-08-10-beam-2.6.0.md
create mode 100644 src/_posts/src/_posts/2018-08-10-beam-2.6.0.md
[beam-site] 04/04: This closes #531
Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mergebot-role pushed a commit to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git
commit 0b58334b64b4fa3c3217b1c82213d35687e3095a
Merge: 6de1aef 77d3348
Author: Mergebot <me...@apache.org>
AuthorDate: Tue Aug 14 19:55:20 2018 +0000
This closes #531
src/contribute/release-guide.md | 383 ++++++++++++++++++++++++++++------------
1 file changed, 266 insertions(+), 117 deletions(-)
[beam-site] 03/04: Addressed Ahmet's comments
Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mergebot-role pushed a commit to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git
commit 77d3348f2d3cd6e332adae21dd1437328e7dd911
Author: Boyuan Zhang <bo...@google.com>
AuthorDate: Tue Aug 14 11:26:02 2018 -0700
Addressed Ahmet's comments
---
src/contribute/release-guide.md | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/src/contribute/release-guide.md b/src/contribute/release-guide.md
index 61e5502..aaffb28 100644
--- a/src/contribute/release-guide.md
+++ b/src/contribute/release-guide.md
@@ -304,7 +304,7 @@ There are 2 ways to trigger a nightly build, either using automation script(reco
./beam/release/src/main/scripts/start_snapshot_build.sh
* Tasks included
- 1. Install [hub](https://github.com/github/hub) within your agreement.
+ 1. Install [hub](https://github.com/github/hub) with your agreement.
1. Touch an empty txt file and commit changes into ```${your remote beam repo}/snapshot_build```
1. Use hub to create a PR against apache:master, which triggers a Jenkins job to build snapshot.
@@ -331,7 +331,7 @@ There are 2 ways to perform this verification, either running automation script(
./beam/release/src/main/scripts/verify_release_build.sh
* Tasks included
- 1. Install ```pip```, ```virtualenv```, ```cython``` and ```/usr/bin/time``` within your agreements.
+ 1. Install ```pip```, ```virtualenv```, ```cython``` and ```/usr/bin/time``` with your agreements.
1. Run ```gradle release build``` against release branch.
* Tasks you need to do manually
@@ -646,7 +646,7 @@ If there are no issues, reply on the vote thread to close the voting. Then, tall
### Run validation tests
All tests listed in this [spreadsheet](https://s.apache.org/beam-release-validation)
-Since there are a bunch of tests, we recommend you running validatoins using automation script. In case of script failure, you can still run all of them manually.
+Since there are a bunch of tests, we recommend you running validations using automation script. In case of script failure, you can still run all of them manually.
#### Run validations using run_rc_validation.sh
* Script: [run_rc_validation.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/run_rc_validation.sh)
[beam-site] 01/04: Add instructions of using automation scripts
Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mergebot-role pushed a commit to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git
commit 33288e94a72db1dee23e861becf6921a16bb2351
Author: Boyuan Zhang <bo...@google.com>
AuthorDate: Wed Aug 8 17:35:59 2018 -0700
Add instructions of using automation scripts
---
src/contribute/release-guide.md | 373 +++++++++++++++++++++++++++-------------
1 file changed, 255 insertions(+), 118 deletions(-)
diff --git a/src/contribute/release-guide.md b/src/contribute/release-guide.md
index a120f0f..73a5840 100644
--- a/src/contribute/release-guide.md
+++ b/src/contribute/release-guide.md
@@ -77,41 +77,59 @@ To prepare for each release, you should audit the project status in the JIRA iss
You need to have a GPG key to sign the release artifacts. Please be aware of the ASF-wide [release signing guidelines](https://www.apache.org/dev/release-signing.html). If you don’t have a GPG key associated with your Apache account, please create one according to the guidelines.
-Get more entropy for creating a GPG key
+There are 2 ways to configure your GPG key for release, either using release automation script(which is recommended),
+or running all commands manually.
- sudo apt-get install rng-tools
- sudo rngd -r /dev/urandom
+##### Use [preparation_before_release.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/preparation_before_release.sh) to setup GPG
+* Usage
+ ```
+ ./beam/release/src/main/scripts/preparation_before_release.sh
+ ```
+* Tasks included
+ 1. Help you create a new GPG key if you want.
+ 1. Configure ```git user.signingkey``` with chosen pubkey.
+ 1. Add chosen pubkey into [dev KEYS](https://dist.apache.org/repos/dist/dev/beam/KEYS) and [release KEYS](https://dist.apache.org/repos/dist/release/beam/KEYS)
+
+ **NOTES**: Only PMC can write into [release repo](https://dist.apache.org/repos/dist/release/beam/).
+ 1. Start GPG agents.
-Create a GPG key
-
- gpg --full-generate-key
+##### Run all commands manually
-Determine your Apache GPG Key and Key ID, as follows:
+* Get more entropy for creating a GPG key
- gpg --list-keys
+ sudo apt-get install rng-tools
+ sudo rngd -r /dev/urandom
-This will list your GPG keys. One of these should reflect your Apache account, for example:
+* Create a GPG key
+
+ gpg --full-generate-key
- --------------------------------------------------
- pub 2048R/845E6689 2016-02-23
- uid Nomen Nescio <an...@apache.org>
- sub 2048R/BA4D50BE 2016-02-23
+* Determine your Apache GPG Key and Key ID, as follows:
+
+ gpg --list-keys
+
+ This will list your GPG keys. One of these should reflect your Apache account, for example:
+
+ --------------------------------------------------
+ pub 2048R/845E6689 2016-02-23
+ uid Nomen Nescio <an...@apache.org>
+ sub 2048R/BA4D50BE 2016-02-23
-Here, the key ID is the 8-digit hex string in the `pub` line: `845E6689`.
+ Here, the key ID is the 8-digit hex string in the `pub` line: `845E6689`.
-Now, add your Apache GPG key to the Beam’s `KEYS` file both in [`dev`](https://dist.apache.org/repos/dist/dev/beam/KEYS) and [`release`](https://dist.apache.org/repos/dist/release/beam/KEYS) repositories at `dist.apache.org`. Follow the instructions listed at the top of these files. (Note: Only PMC members have write access to the release repository. If you end up getting 403 errors ask on the mailing list for assistance.)
+ Now, add your Apache GPG key to the Beam’s `KEYS` file both in [`dev`](https://dist.apache.org/repos/dist/dev/beam/KEYS) and [`release`](https://dist.apache.org/repos/dist/release/beam/KEYS) repositories at `dist.apache.org`. Follow the instructions listed at the top of these files. (Note: Only PMC members have write access to the release repository. If you end up getting 403 errors ask on the mailing list for assistance.)
-Configure `git` to use this key when signing code by giving it your key ID, as follows:
+* Configure `git` to use this key when signing code by giving it your key ID, as follows:
- git config --global user.signingkey 845E6689
+ git config --global user.signingkey 845E6689
-You may drop the `--global` option if you’d prefer to use this key for the current repository only.
+ You may drop the `--global` option if you’d prefer to use this key for the current repository only.
-You may wish to start `gpg-agent` to unlock your GPG key only once using your passphrase. Otherwise, you may need to enter this passphrase hundreds of times. The setup for `gpg-agent` varies based on operating system, but may be something like this:
+* Start GPG agent in order to unlock your GPG key
- eval $(gpg-agent --daemon --no-grab --write-env-file $HOME/.gpg-agent-info)
- export GPG_TTY=$(tty)
- export GPG_AGENT_INFO
+ eval $(gpg-agent --daemon --no-grab --write-env-file $HOME/.gpg-agent-info)
+ export GPG_TTY=$(tty)
+ export GPG_AGENT_INFO
#### Access to Apache Nexus repository
@@ -181,135 +199,201 @@ You should verify that the issues listed automatically by JIRA are appropriate t
Adjust any of the above properties to the improve clarity and presentation of the Release Notes.
-### Verify that a Release Build Works
+### Create a release branch in apache/beam repository
-Pre-installation for python build
-* Install pip
-
- ```
- curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
- python get-pip.py
- ```
-* Install virtualenv
+Attention: Only committer has permission to create release branch in apache/beam.
- ```
- pip install --upgrade virtualenv
- ```
-* Cython
+Release candidates are built from a release branch. As a final step in preparation for the release, you should create the release branch, push it to the Apache code repository, and update version information on the original branch.
+
+There are 2 ways to cut a release branch: either running automation script(recommended), or running all commands manually.
+#### Use [cut_release_branch.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/cut_release_branch.sh) to cut a release branch
+* Usage
```
- sudo pip install cython
- sudo apt-get install gcc
- sudo apt-get install python-dev
+ # Cut a release branch
+ ./beam/release/src/main/scripts/cut_release_branch.sh
+ --release= ${RELEASE_VERSION}
+ --next_release=${NEXT_VERSION}
+
+ # Show help page
+ ./beam/release/src/main/scripts/cut_release_branch.sh -h
```
-Make sure your ```time``` alias to ```/usr/bin/time```, if not:
+* Tasks included
+ 1. Create release-${RELEASE_VERSION} branch locally.
+ 1. Change and commit dev versoin number in master branch:
+
+ [BeamModulePlugin.groovy](https://github.com/apache/beam/blob/e8abafe360e126818fe80ae0f6075e71f0fc227d/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L209),
+ [gradle.properties](https://github.com/apache/beam/blob/e8abafe360e126818fe80ae0f6075e71f0fc227d/gradle.properties#L25),
+ [version.py](https://github.com/apache/beam/blob/e8abafe360e126818fe80ae0f6075e71f0fc227d/sdks/python/apache_beam/version.py#L21)
+ 1. Change and commit version number in release branch:
+
+ [version.py](https://github.com/apache/beam/blob/release-2.6.0/sdks/python/apache_beam/version.py#L21),
+ [build.gradle](https://github.com/apache/beam/blob/release-2.6.0/runners/google-cloud-dataflow-java/build.gradle#L39)
+
+#### Run all steps manually
+* Checkout working branch
+
+ Check out the version of the codebase from which you start the release. For a new minor or major release, this may be `HEAD` of the `master` branch. To build a hotfix/incremental release, instead of the `master` branch, use the release tag of the release being patched. (Please make sure your cloned repository is up-to-date before starting.)
-```
-sudo apt-get install time
-alias time='/usr/bin/time'
-```
+ git checkout <master branch OR release tag>
-Run gradle release build
+ **NOTE**: If you are doing an incremental/hotfix release (e.g. 2.5.1), please check out the previous release tag, rather than the master branch.
-* Clean current workspace
+* Set up environment variables
- ```
- git clean -fdx
- ./gradlew clean
- ```
+ Set up a few environment variables to simplify Maven commands that follow. (We use `bash` Unix syntax in this guide.)
-* Unlock the secret key
- ```
- gpg --output ~/doc.sig --sign ~/.bashrc
- ```
+ RELEASE=2.5.0
+ NEXT_VERSION_IN_BASE_BRANCH=2.6.0
+ BRANCH=release-${RELEASE}
-* Run build command
- ```
- ./gradlew build -PisRelease --no-parallel --scan --stacktrace
- ```
-
-### Update and Verify Javadoc
+ Version represents the release currently underway, while next version specifies the anticipated next version to be released from that branch. Normally, 1.2.0 is followed by 1.3.0, while 1.2.3 is followed by 1.2.4.
-The build with `-PisRelease` creates the combined Javadoc for the release in `sdks/java/javadoc`.
+ **NOTE**: Only if you are doing an incremental/hotfix release (e.g. 2.5.1), please check out the previous release tag, before running the following instructions:
-The file `sdks/java/javadoc/ant.xml` file contains a list of modules to include
-in and exclude, plus a list of offline URLs that populate links from Beam's
-Javadoc to the Javadoc for other modules that Beam depends on.
+ BASE_RELEASE=2.5.0
+ RELEASE=2.5.1
+ NEXT_VERSION_IN_BASE_BRANCH=2.6.0
+ git checkout tags/${BASE_RELEASE}
-* Confirm that new modules added since the last release have been added to the
- inclusion list as appropriate.
+* Create release branch locally
+
+ git branch ${BRANCH}
-* Confirm that the excluded package list is up to date.
+* Update version files in the master branch.
+
+ # Now change the version in existing gradle files, and Python files
+ sed -i -e "s/'${RELEASE}'/'${NEXT_VERSION_IN_BASE_BRANCH}'/g" build_rules.gradle
+ sed -i -e "s/${RELEASE}/${NEXT_VERSION_IN_BASE_BRANCH}/g" gradle.properties
+ sed -i -e "s/${RELEASE}/${NEXT_VERSION_IN_BASE_BRANCH}/g" sdks/python/apache_beam/version.py
+
+ # Save changes in master branch
+ git add gradle.properties build_rules.gradle sdks/python/apache_beam/version.py
+ git commit -m "Moving to ${NEXT_VERSION_IN_BASE_BRANCH}-SNAPSHOT on master branch."
-* Verify the version numbers for offline links match the versions used by Beam. If
- the version number has changed, download a new version of the corresponding
- `<module>-docs/package-list` file.
+* Check out the release branch.
-### Create a release branch in apache/beam repository
+ git checkout ${BRANCH}
+
-Attention: Only committer has permission to create release branch in apache/beam.
+* Update version files in release branch
+
+ DEV=${RELEASE}.dev
+ sed -i -e "s/${DEV}/${RELEASE}/g" sdks/python/apache_beam/version.py
+ sed -i -e "s/'beam-master-.*'/'beam-${RELEASE}'/g" runners/google-cloud-dataflow-java/build.gradle
-Release candidates are built from a release branch. As a final step in preparation for the release, you should create the release branch, push it to the Apache code repository, and update version information on the original branch.
-Check out the version of the codebase from which you start the release. For a new minor or major release, this may be `HEAD` of the `master` branch. To build a hotfix/incremental release, instead of the `master` branch, use the release tag of the release being patched. (Please make sure your cloned repository is up-to-date before starting.)
+### Start a snapshot build
- git checkout <master branch OR release tag>
+Start a build of [the nightly snapshot](https://builds.apache.org/view/A-D/view/Beam/job/beam_Release_NightlySnapshot/) against master branch.
+Some processes, including our archetype tests, rely on having a live SNAPSHOT of the current version
+from the `master` branch. Once the release branch is cut, these SNAPSHOT versions are no longer found,
+so builds will be broken until a new snapshot is available.
-**NOTE**: If you are doing an incremental/hotfix release (e.g. 2.5.1), please check out the previous release tag, rather than the master branch.
+There are 2 ways to trigger a nightly build, either using automation script(recommended), or perform all operations manually.
-Set up a few environment variables to simplify Maven commands that follow. (We use `bash` Unix syntax in this guide.)
+#### Run [start_snapshot_build.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/start_snapshot_build.sh) to trigger build
+* Usage
+
+ ./beam/release/src/main/scripts/start_snapshot_build.sh
- RELEASE=2.5.0
- NEXT_VERSION_IN_BASE_BRANCH=2.6.0
- BRANCH=release-${RELEASE}
+* Tasks included
+ 1. Install [hub](https://github.com/github/hub) within your agreement.
+ 1. Touch an empty txt file and commit changes into ```${your remote beam repo}/snapshot_build```
+ 1. Use hub to create a PR against apache:master, which triggers a Jenkins job to build snapshot.
+
+* Tasks you need to do manually
+ 1. Check whether the Jenkins job gets triggered. If not, please comment ```Run Gradle Publish``` into the generated PR.
+ 1. After verifying build succeeded, you need to close PR manually.
+
+#### Do all operations manually
-Version represents the release currently underway, while next version specifies the anticipated next version to be released from that branch. Normally, 1.2.0 is followed by 1.3.0, while 1.2.3 is followed by 1.2.4.
+* Find one PR against apache:master in beam.
+* Comment ```Run Gradle Publish``` in this pull request to trigger build.
+* Verify that build successes.
-**NOTE**: Only if you are doing an incremental/hotfix release (e.g. 2.5.1), please check out the previous release tag, before running the following instructions:
- BASE_RELEASE=2.5.0
- RELEASE=2.5.1
- NEXT_VERSION_IN_BASE_BRANCH=2.6.0
- git checkout tags/${BASE_RELEASE}
+### Verify that a Release Build Works
-Create a new branch, and update version files in the master branch.
+There are 2 ways to perform this verification, either running automation script(recommended), or running all commands manually.
- git branch ${BRANCH}
+#### Run [verify_release_build.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/verify_release_build.sh) to verity a release build
+* Usage
+
+ ./beam/release/src/main/scripts/verify_release_build.sh
- # Now change the version in existing gradle files, and Python files
- sed -i -e "s/'${RELEASE}'/'${NEXT_VERSION_IN_BASE_BRANCH}'/g" build_rules.gradle
- sed -i -e "s/${RELEASE}/${NEXT_VERSION_IN_BASE_BRANCH}/g" gradle.properties
- sed -i -e "s/${RELEASE}/${NEXT_VERSION_IN_BASE_BRANCH}/g" sdks/python/apache_beam/version.py
+* Tasks included
+ 1. Install ```pip```, ```virtualenv```, ```cython``` and ```/usr/bin/time``` within your agreements.
+ 1. Run ```gradle release build``` against release branch.
+
+* Tasks you need to do manually
+ 1. Check the build result.
+ 1. If build failed, scan log will contain all failures.
+ 1. You should stabilize the release branch until release build succeeded.
+
+#### Run all commands manually
+* Pre-installation for python build
+ 1. Install pip
+
+ ```
+ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
+ python get-pip.py
+ ```
+ 1. Install virtualenv
- # Save changes in master branch
- git add gradle.properties build_rules.gradle sdks/python/apache_beam/version.py
- git commit -m "Moving to ${NEXT_VERSION_IN_BASE_BRANCH}-SNAPSHOT on master branch."
+ ```
+ pip install --upgrade virtualenv
+ ```
+ 1. Cython
-Check out the release branch.
+ ```
+ sudo pip install cython
+ sudo apt-get install gcc
+ sudo apt-get install python-dev
+ ```
+ 1. Make sure your ```time``` alias to ```/usr/bin/time```, if not:
- git checkout ${BRANCH}
-
-The rest of this guide assumes that commands are run in the root of a repository on `${BRANCH}` with the above environment variables set.
+ ```
+ sudo apt-get install time
+ alias time='/usr/bin/time'
+ ```
-### Update the Python SDK version
+* Run gradle release build
-Update [sdks/python/apache_beam/version.py](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/version.py) in both master branch and release branch.
+ 1. Clean current workspace
-* In the release branch, update the Python SDK version to the release version(e.g. `2.5.0-dev` to `2.5.0`).
+ ```
+ git clean -fdx
+ ./gradlew clean
+ ```
-### Update release specific configurations
+ 1. Unlock the secret key
+ ```
+ gpg --output ~/doc.sig --sign ~/.bashrc
+ ```
-Update Java runner specific configurations in release branch:
-* [beam/runners/google-cloud-dataflow-java/build.gradle](https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/build.gradle): change value of 'dataflow.container_version' to 'beam-release_version_number'(e.g, 'beam-master-20180601' to 'beam-2.5.0')
+ 1. Run build command
+ ```
+ ./gradlew build -PisRelease --no-parallel --scan --stacktrace --continue
+ ```
+
+### Update and Verify Javadoc
-### Start a snapshot build
+The build with `-PisRelease` creates the combined Javadoc for the release in `sdks/java/javadoc`.
-Start a build of [the nightly snapshot](https://builds.apache.org/view/A-D/view/Beam/job/beam_Release_NightlySnapshot/) against master branch.
-Some processes, including our archetype tests, rely on having a live SNAPSHOT of the current version
-from the `master` branch. Once the release branch is cut, these SNAPSHOT versions are no longer found,
-so builds will be broken until a new snapshot is available.
+The file `sdks/java/javadoc/ant.xml` file contains a list of modules to include
+in and exclude, plus a list of offline URLs that populate links from Beam's
+Javadoc to the Javadoc for other modules that Beam depends on.
+
+* Confirm that new modules added since the last release have been added to the
+ inclusion list as appropriate.
+
+* Confirm that the excluded package list is up to date.
+
+* Verify the version numbers for offline links match the versions used by Beam. If
+ the version number has changed, download a new version of the corresponding
+ `<module>-docs/package-list` file.
-Comment ```Run Gradle Publish``` in one pull request to trigger build.
### Checklist to proceed to the next step
@@ -331,7 +415,31 @@ Comment ```Run Gradle Publish``` in one pull request to trigger build.
The core of the release process is the build-vote-fix cycle. Each cycle produces one release candidate. The Release Manager repeats this cycle until the community approves one release candidate, which is then finalized.
-### Build and stage Java artifacts with Gradle
+For this step, we recommend you using automation script to create a RC, but you still can perform all steps manually if you want.
+
+### Run [build_release_candidate.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/build_release_candidate.sh) to create RC
+* Usage
+
+ ./beam/release/src/main/scripts/build_release_candidate.sh
+
+* Tasks included
+ 1. Run gradle release to create rc tag and push source release into github repo.
+ 1. Run gradle publish to push java artifacts into Maven staging repo.
+ 1. Stage source release into dist.apache.org dev [repo](https://dist.apache.org/repos/dist/dev/beam/).
+ 1. Stage,sign and hash python binaries into dist.apache.ord dev repo python dir
+ 1. Create a PR to update beam-site, changes includes:
+ * Copy python doc into beam-site
+ * Copy java doc into beam-site
+ * Update release version into [_config.yml](https://github.com/apache/beam-site/blob/asf-site/_config.yml).
+
+* Tasks you need to do manually
+ 1. Add new release into src/get-started/downloads.md
+ 1. Update last release download links in src/get-started/downloads.md
+ 1. Update the Pydoc link on this page to point to the new version (in src/documentation/sdks/pydoc/current.md.
+
+### Run all steps manually
+
+#### Build and stage Java artifacts with Gradle
Set up a few environment variables to simplify the commands that follow. These identify the release candidate being built, and the branch where you will stage files. Start with `RC_NUM` equal to `1` and increment it for each candidate.
@@ -356,7 +464,7 @@ Review all staged artifacts. They should contain all relevant parts for each mod
Close the staging repository on Apache Nexus. When prompted for a description, enter “Apache Beam, version X, release candidate Y”.
-### Stage source release on dist.apache.org
+#### Stage source release on dist.apache.org
Attention: Only committer has permissions to perform following steps.
@@ -388,7 +496,7 @@ Copy the source release to the dev repository of `dist.apache.org`.
1. Verify that files are [present](https://dist.apache.org/repos/dist/dev/beam).
-### Stage python binaries on dist.apache.org
+#### Stage python binaries on dist.apache.org
Build python binaries in release branch in sdks/python dir.
@@ -411,7 +519,7 @@ Staging binaries
Verify that files are [present](https://dist.apache.org/repos/dist/dev/beam).
-### Build the Pydoc API reference
+#### Build the Pydoc API reference
Make sure you have ```tox``` installed:
@@ -425,7 +533,7 @@ cd sdks/python && tox -e docs
```
By default the Pydoc is generated in `sdks/python/target/docs/_build`. Let `${PYDOC_ROOT}` be the absolute path to `_build`.
-### Propose a pull request for website updates
+#### Propose a pull request for website updates
The final step of building the candidate is to propose a website pull request.
@@ -444,7 +552,7 @@ Add the new Javadoc to [SDK API Reference page]({{ site.baseurl }}/documentation
* Set up the necessary git commands to account for the new and deleted files from the javadoc.
* Update the Javadoc link on this page to point to the new version (in `src/documentation/sdks/javadoc/current.md`).
-#### Create Pydoc
+##### Create Pydoc
Add the new Pydoc to [SDK API Reference page]({{ site.baseurl }}/documentation/sdks/pydoc/) page, as follows:
* Copy the generated Pydoc into the website repository: `cp -r ${PYDOC_ROOT} src/documentation/sdks/pydoc/${RELEASE}`.
@@ -453,7 +561,7 @@ Add the new Pydoc to [SDK API Reference page]({{ site.baseurl }}/documentation/s
Finally, propose a pull request with these changes. (Don’t merge before finalizing the release.)
-### Checklist to proceed to the next step
+#### Checklist to proceed to the next step
1. Maven artifacts deployed to the staging repository of [repository.apache.org](https://repository.apache.org/content/repositories/)
1. Source distribution deployed to the dev repository of [dist.apache.org](https://dist.apache.org/repos/dist/dev/beam/)
@@ -528,6 +636,34 @@ If there are no issues, reply on the vote thread to close the voting. Then, tall
### Run validation tests
All tests listed in this [spreadsheet](https://s.apache.org/beam-release-validation)
+Since there are a bunch of tests, we recommend you running validatoins using automation script. In case of script failure, you can still run all of them manually.
+
+#### Run validations using [run_rc_validation.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/run_rc_validation.sh)
+* Usage
+
+ ./beam/release/src/main/scripts/run_rc_validation.sh
+
+* Tasks included
+ 1. Run Java quickstart with Direct Runner, Apex local runner, Flink local runner, Spark local runner and Dataflow runner.
+ 1. Run Java Mobile Games(UserScore, HourlyTeamScore, Leaderboard) with Dataflow runner.
+ 1. Create a PR against apache:master to trigger python validation job, including
+ * Python quickstart in batch and streaming mode with direct runner and Dataflow runner.
+ * Python Mobile Games(UserScore, HourlyTeamScore) with direct runner and Dataflow runner.
+ 1. Run Python Streaming MobileGames, includes
+ * Start a new terminal to run Java Pubsub injector.
+ * Start a new terminal to run python LeaderBoard with Direct Runner.
+ * Start a new terminal to run python LeaderBoard with Dataflow Runner.
+ * Start a new terminal to run python GameStats with Direct Runner.
+ * Start a new terminal to run python GameStats with Dataflow Runner.
+
+* Tasks you need to do manually
+ 1. Check whether validations succeed by following console output instructions.
+ 1. Terminate streaming jobs and java injector.
+ 1. Sign up [spreadsheet](https://s.apache.org/beam-release-validation).
+ 1. Vote in the release thread.
+
+#### Run validations manually
+
_Note_: -Prepourl and -Pver can be found in the RC vote email sent by Release Manager.
* Java Quickstart Validation
@@ -590,7 +726,7 @@ _Note_: -Prepourl and -Pver can be found in the RC vote email sent by Release Ma
-Prepourl=https://repository.apache.org/content/repositories/orgapachebeam-${KEY} \
-Pver= ${RELEASE_VERSION}\
-PgcpProject=${YOUR_GCP_PROJECT} \
- -PgcsBucket=gs://dataflow-samples/game/gaming_data1.csv \
+ -PgcsBucket=${YOUR_GCP_BUCKET} \
-PbqDataset=${YOUR_DATASET} -PpubsubTopic=${YOUR_PROJECT_PUBSUB_TOPIC}
```
* Python Quickstart(batch & streaming), MobileGame(UserScore, HourlyTeamScore)
@@ -738,6 +874,7 @@ _Note_: -Prepourl and -Pver can be found in the RC vote email sent by Release Ma
* Goto your BigQuery console and check whether your ${USER}_test has game_stats_teams and game_stats_sessions table.
* bq head -n 10 ${USER}_test.game_stats_teams
* bq head -n 10 ${USER}_test.game_stats_sessions
+
### Checklist to proceed to the finalization step
1. Community votes to release the proposed candidate, with at least three approving PMC votes
[beam-site] 02/04: Fix broken tests
Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mergebot-role pushed a commit to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git
commit 115d974675a2d93c438fae1ca80a39bb4a4143a3
Author: Boyuan Zhang <bo...@google.com>
AuthorDate: Fri Aug 10 14:38:57 2018 -0700
Fix broken tests
---
src/contribute/release-guide.md | 24 ++++++++++++++++++------
1 file changed, 18 insertions(+), 6 deletions(-)
diff --git a/src/contribute/release-guide.md b/src/contribute/release-guide.md
index 73a5840..61e5502 100644
--- a/src/contribute/release-guide.md
+++ b/src/contribute/release-guide.md
@@ -80,7 +80,9 @@ You need to have a GPG key to sign the release artifacts. Please be aware of the
There are 2 ways to configure your GPG key for release, either using release automation script(which is recommended),
or running all commands manually.
-##### Use [preparation_before_release.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/preparation_before_release.sh) to setup GPG
+##### Use preparation_before_release.sh to setup GPG
+* Script: [preparation_before_release.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/preparation_before_release.sh)
+
* Usage
```
./beam/release/src/main/scripts/preparation_before_release.sh
@@ -207,7 +209,9 @@ Release candidates are built from a release branch. As a final step in preparati
There are 2 ways to cut a release branch: either running automation script(recommended), or running all commands manually.
-#### Use [cut_release_branch.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/cut_release_branch.sh) to cut a release branch
+#### Use cut_release_branch.sh to cut a release branch
+* Script: [cut_release_branch.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/cut_release_branch.sh)
+
* Usage
```
# Cut a release branch
@@ -292,7 +296,9 @@ so builds will be broken until a new snapshot is available.
There are 2 ways to trigger a nightly build, either using automation script(recommended), or perform all operations manually.
-#### Run [start_snapshot_build.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/start_snapshot_build.sh) to trigger build
+#### Run start_snapshot_build.sh to trigger build
+* Script: [start_snapshot_build.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/start_snapshot_build.sh)
+
* Usage
./beam/release/src/main/scripts/start_snapshot_build.sh
@@ -317,7 +323,9 @@ There are 2 ways to trigger a nightly build, either using automation script(reco
There are 2 ways to perform this verification, either running automation script(recommended), or running all commands manually.
-#### Run [verify_release_build.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/verify_release_build.sh) to verity a release build
+#### Run verify_release_build.sh to verity a release build
+* Script: [verify_release_build.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/verify_release_build.sh)
+
* Usage
./beam/release/src/main/scripts/verify_release_build.sh
@@ -417,7 +425,9 @@ The core of the release process is the build-vote-fix cycle. Each cycle produces
For this step, we recommend you using automation script to create a RC, but you still can perform all steps manually if you want.
-### Run [build_release_candidate.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/build_release_candidate.sh) to create RC
+### Run build_release_candidate.sh to create RC
+* Script: [build_release_candidate.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/build_release_candidate.sh)
+
* Usage
./beam/release/src/main/scripts/build_release_candidate.sh
@@ -638,7 +648,9 @@ All tests listed in this [spreadsheet](https://s.apache.org/beam-release-validat
Since there are a bunch of tests, we recommend you running validatoins using automation script. In case of script failure, you can still run all of them manually.
-#### Run validations using [run_rc_validation.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/run_rc_validation.sh)
+#### Run validations using run_rc_validation.sh
+* Script: [run_rc_validation.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/run_rc_validation.sh)
+
* Usage
./beam/release/src/main/scripts/run_rc_validation.sh