You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by da...@apache.org on 2016/11/05 00:52:25 UTC
[1/4] incubator-beam-site git commit: Add missing site.baseurls
Repository: incubator-beam-site
Updated Branches:
refs/heads/asf-site e96b07f54 -> 228a6c91d
Add missing site.baseurls
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/a7c71d23
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/tree/a7c71d23
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/diff/a7c71d23
Branch: refs/heads/asf-site
Commit: a7c71d23932162c87774ffbfcff81260c3ffaedb
Parents: e96b07f
Author: Dan Halperin <dh...@google.com>
Authored: Fri Nov 4 12:34:04 2016 -0700
Committer: Dan Halperin <dh...@google.com>
Committed: Fri Nov 4 17:46:00 2016 -0700
----------------------------------------------------------------------
_includes/footer.html | 2 +-
_includes/head.html | 2 +-
_includes/header.html | 2 +-
_posts/2016-08-03-six-months.md | 2 +-
.../2016-10-12-strata-hadoop-world-and-beam.md | 2 +-
_posts/2016-10-20-test-stream.md | 6 ++---
blog/index.md | 2 +-
coming-soon.md | 2 +-
contribute/contribution-guide.md | 8 +++----
contribute/index.md | 2 +-
contribute/logos.md | 9 ++++----
contribute/release-guide.md | 14 ++++++------
contribute/team.md | 5 ++--
contribute/work-in-progress.md | 6 ++---
documentation/index.md | 7 +++---
documentation/pipelines/create-your-pipeline.md | 2 +-
documentation/pipelines/design-your-pipeline.md | 2 +-
documentation/pipelines/test-your-pipeline.md | 2 +-
documentation/programming-guide.md | 5 ++--
documentation/resources.md | 4 ++--
documentation/runners/capability-materix.md | 5 ++--
documentation/runners/dataflow.md | 2 +-
documentation/runners/direct.md | 2 +-
documentation/runners/flink.md | 2 +-
documentation/runners/index.md | 2 +-
documentation/runners/spark.md | 2 +-
documentation/sdks/index.md | 2 +-
documentation/sdks/java.md | 6 ++---
documentation/sdks/javadoc/index.md | 2 +-
get-started/beam-overview.md | 7 +++---
get-started/downloads.md | 2 +-
get-started/index.md | 9 ++++----
get-started/mobile-gaming-example.md | 24 ++++++++++----------
get-started/quickstart.md | 5 ++--
get-started/releases.md | 5 ++--
get-started/support.md | 3 +++
get-started/wordcount-example.md | 2 +-
index.md | 6 ++---
38 files changed, 93 insertions(+), 81 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/_includes/footer.html
----------------------------------------------------------------------
diff --git a/_includes/footer.html b/_includes/footer.html
index 922c68e..85830ef 100644
--- a/_includes/footer.html
+++ b/_includes/footer.html
@@ -4,7 +4,7 @@
<footer>
<p class="text-center">© Copyright 2016
<a href="http://www.apache.org">The Apache Software Foundation.</a> All Rights Reserved.</p>
- <p class="text-center"><a href="/privacy_policy">Privacy Policy</a> |
+ <p class="text-center"><a href="{{ site.baseurl }}/privacy_policy">Privacy Policy</a> |
<a href="{{ "/feed.xml" | prepend: site.baseurl }}">RSS Feed</a></p>
</footer>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/_includes/head.html
----------------------------------------------------------------------
diff --git a/_includes/head.html b/_includes/head.html
index 02c7727..80f1f20 100644
--- a/_includes/head.html
+++ b/_includes/head.html
@@ -23,5 +23,5 @@
ga('send', 'pageview');
</script>
- <link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
+ <link rel="shortcut icon" type="image/x-icon" href="{{ site.baseurl }}/images/favicon.ico">
</head>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/_includes/header.html
----------------------------------------------------------------------
diff --git a/_includes/header.html b/_includes/header.html
index 97d9399..f70bcee 100644
--- a/_includes/header.html
+++ b/_includes/header.html
@@ -1,7 +1,7 @@
<nav class="navbar navbar-default navbar-fixed-top">
<div class="container">
<div class="navbar-header">
- <a href="/" class="navbar-brand" >
+ <a href="{{ site.baseurl }}/" class="navbar-brand" >
<img alt="Brand" style="height: 25px" src="{{ "/images/beam_logo_navbar.png" | prepend: site.baseurl }}">
</a>
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/_posts/2016-08-03-six-months.md
----------------------------------------------------------------------
diff --git a/_posts/2016-08-03-six-months.md b/_posts/2016-08-03-six-months.md
index 1141eaf..73305e4 100644
--- a/_posts/2016-08-03-six-months.md
+++ b/_posts/2016-08-03-six-months.md
@@ -39,5 +39,5 @@ But perhaps most importantly, we\u2019re committed to building an involved, welcomi
* Had multiple talks on Beam at venues including ApacheCon, Hadoop Summit, Kafka Summit, JBCN Barcelona, and Strata.
* Presented at multiple existing meetups and are starting to organize some of our own.
-While it\u2019s nice to reflect back on all we\u2019ve done, we\u2019re working full _stream_ ahead towards a stable release and graduation from incubator. And we\u2019d love your help -- join the [mailing lists](http://beam.incubator.apache.org/use/mailing-lists/), check out the [contribution guide](http://beam.incubator.apache.org/contribute/contribution-guide/), and grab a [starter task](https://issues.apache.org/jira/browse/BEAM-520?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20in%20(newbie%2C%20starter)) from Jira!
+While it\u2019s nice to reflect back on all we\u2019ve done, we\u2019re working full _stream_ ahead towards a stable release and graduation from incubator. And we\u2019d love your help -- join the [mailing lists]({{ site.baseurl }}/use/mailing-lists/), check out the [contribution guide]({{ site.baseurl }}/contribute/contribution-guide/), and grab a [starter task](https://issues.apache.org/jira/browse/BEAM-520?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20in%20(newbie%2C%20starter)) from Jira!
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/_posts/2016-10-12-strata-hadoop-world-and-beam.md
----------------------------------------------------------------------
diff --git a/_posts/2016-10-12-strata-hadoop-world-and-beam.md b/_posts/2016-10-12-strata-hadoop-world-and-beam.md
index acd48d7..f190762 100644
--- a/_posts/2016-10-12-strata-hadoop-world-and-beam.md
+++ b/_posts/2016-10-12-strata-hadoop-world-and-beam.md
@@ -28,5 +28,5 @@ On management and thought leader side, Beam went from \u201cwhat\u2019s Beam?\u201d at pr
We rounded out the tutorial with live demonstrations of Beam running on Apache Spark, Apache Flink, the local runner, and DataFlow runner. Then, we brought in the big brainpower and had a Q and A session.
-If you\u2019re attending a conference, we encourage you to look for a Beam session. If you want to use these materials to give your own Beam talk or tutorial, we\u2019re happy to help you. In addition to this tutorial, we have [other presentation materials](/learn/presentation-materials/). You can reach out to us on the [user mailing list](http://beam.incubator.apache.org/use/mailing-lists/).
+If you\u2019re attending a conference, we encourage you to look for a Beam session. If you want to use these materials to give your own Beam talk or tutorial, we\u2019re happy to help you. In addition to this tutorial, we have [other presentation materials]({{ site.baseurl }}/learn/presentation-materials/). You can reach out to us on the [user mailing list]({{ site.baseurl }}/use/mailing-lists/).
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/_posts/2016-10-20-test-stream.md
----------------------------------------------------------------------
diff --git a/_posts/2016-10-20-test-stream.md b/_posts/2016-10-20-test-stream.md
index 8b7eee7..e2e52de 100644
--- a/_posts/2016-10-20-test-stream.md
+++ b/_posts/2016-10-20-test-stream.md
@@ -47,7 +47,7 @@ from the Mobile Gaming example series.
[LeaderBoard](https://github.com/apache/incubator-beam/blob/master/examples/java8/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java#L177)
is part of the [Beam mobile gaming examples](https://github.com/apache/incubator-beam/tree/master/examples/java8/src/main/java/org/apache/beam/examples/complete/game)
-(and [walkthroughs](http://beam.incubator.apache.org/use/walkthroughs/))
+(and [walkthroughs]({{ site.baseurl }}/get-started/mobile-gaming-example/))
which produces a continuous accounting of user and team scores. User scores are
calculated over the lifetime of the program, while team scores are calculated
within fixed windows with a default duration of one hour. The LeaderBoard
@@ -60,7 +60,7 @@ be controlled within a test.
## Writing Deterministic Tests to Emulate Nondeterminism
The Beam testing infrastructure provides the
-[PAssert](http://beam.incubator.apache.org/learn/sdks/javadoc/0.2.0-incubating/)
+[PAssert]({{ site.baseurl }}/documentation/sdks/javadoc/0.2.0-incubating/)
methods, which assert properties about the contents of a PCollection from within
a pipeline. We have expanded this infrastructure to include
[TestStream](https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/TestStream.java),
@@ -306,4 +306,4 @@ The addition of TestStream alongside window and pane-specific matchers in PAsser
has enabled the testing of Pipelines which produce speculative and late panes.
This permits tests for all styles of pipeline to be expressed directly within the
Java SDK. If you have questions or comments, we\u2019d love to hear them on the
-[mailing lists](http://beam.incubator.apache.org/use/mailing-lists/).
+[mailing lists]({{ site.baseurl }}/use/mailing-lists/).
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/blog/index.md
----------------------------------------------------------------------
diff --git a/blog/index.md b/blog/index.md
index 0334a17..cda0cdd 100644
--- a/blog/index.md
+++ b/blog/index.md
@@ -26,7 +26,7 @@ for the project.
{% endcapture %}
{% if excerpt_words != content_words %}
<p>
-<a class="btn btn-default btn-sm" href="{{ post.url }}" role="button">
+<a class="btn btn-default btn-sm" href="{{ post.url | prepend: site.baseurl }}" role="button">
Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"></span>
</a>
</p>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/coming-soon.md
----------------------------------------------------------------------
diff --git a/coming-soon.md b/coming-soon.md
index 6054d3a..e77c753 100644
--- a/coming-soon.md
+++ b/coming-soon.md
@@ -6,4 +6,4 @@ layout: default
You've reached a page that's still in draft, or otherwise being developed! Please bear with us as we improve the documentation for Apache Beam.
-[Go Back](/) to the main Beam site.
+[Go Back]({{ site.baseurl }}/) to the main Beam site.
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/contribute/contribution-guide.md
----------------------------------------------------------------------
diff --git a/contribute/contribution-guide.md b/contribute/contribution-guide.md
index a7c82b1..4635e91 100644
--- a/contribute/contribution-guide.md
+++ b/contribute/contribution-guide.md
@@ -18,7 +18,7 @@ We use a review-then-commit workflow in Beam for all contributions.
**For larger contributions or those that affect multiple components:**
-1. **Engage**: We encourage you to work with the Beam community on the [Apache JIRA issue tracker](https://issues.apache.org/jira/browse/BEAM) and [developer\u2019s mailing list](http://beam.incubator.apache.org/use/mailing-lists/) to identify good areas for contribution.
+1. **Engage**: We encourage you to work with the Beam community on the [Apache JIRA issue tracker](https://issues.apache.org/jira/browse/BEAM) and [developer\u2019s mailing list]({{ site.baseurl }}/use/mailing-lists/) to identify good areas for contribution.
1. **Design:** More complicated contributions will likely benefit from some early discussion in order to scope and design them well.
**For all contributions:**
@@ -34,7 +34,7 @@ We look forward to working with you!
### Mailing list(s)
We discuss design and implementation issues on dev@beam.incubator.apache.org mailing list, which is archived [here](https://lists.apache.org/list.html?dev@beam.apache.org). Join by emailing [`dev-subscribe@beam.incubator.apache.org`](mailto:dev-subscribe@beam.incubator.apache.org).
-If interested, you can also join the other [mailing lists](http://beam.incubator.apache.org/use/mailing-lists/) too.
+If interested, you can also join the other [mailing lists]({{ site.baseurl }}/use/mailing-lists/) too.
### Apache JIRA
We use [Apache JIRA](https://issues.apache.org/jira/browse/BEAM) as an issue tracking and project management tool, as well as a way to communicate among a very diverse and distributed set of contributors. To be able to gather feedback, avoid frustration, and avoid duplicated efforts all Beam-related work should be tracked there.
@@ -314,10 +314,10 @@ Some larger features are developed on a feature branch before being merged into
#### Developing
-To contribute code on a feature branch, use the same process as above, but replace `master` with the [name of the branch](/contribute/work-in-progress/#feature-branches).
+To contribute code on a feature branch, use the same process as above, but replace `master` with the [name of the branch]({{ site.baseurl }}/contribute/work-in-progress/#feature-branches).
Since feature branches are often used for new components, you may find that
-there is no [committer](/project/team/) familiar with all the details of the
+there is no [committer]({{ site.baseurl }}/project/team/) familiar with all the details of the
new language or runner. In that case, consider asking someone else familiar
with the technology to do an initial review before looping in a committer for a
final review and merge.
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/contribute/index.md
----------------------------------------------------------------------
diff --git a/contribute/index.md b/contribute/index.md
index 6d8f99c..9bfda4a 100644
--- a/contribute/index.md
+++ b/contribute/index.md
@@ -20,4 +20,4 @@ Resources for contributing to Beam, including mailing lists, the main GitHub rep
* [Mailing Lists]({{ site.baseurl }}/use/mailing-lists/)
* [Source Repository]({{ site.baseurl }}/contribute/source-repository)
-* [Issue Tracking]({{ site.baseurl }}/use/issue-tracking/)
\ No newline at end of file
+* [Issue Tracking]({{ site.baseurl }}/use/issue-tracking/)
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/contribute/logos.md
----------------------------------------------------------------------
diff --git a/contribute/logos.md b/contribute/logos.md
index fa42c57..0aeccdd 100644
--- a/contribute/logos.md
+++ b/contribute/logos.md
@@ -2,8 +2,9 @@
layout: default
title: "Beam Logos"
permalink: /contribute/logos/
-redirect-from: /project/logos/
-redirect_from: /material/
+redirect_from:
+ - /project/logos/
+ - /material/
---
# Apache Beam Logos
@@ -47,11 +48,11 @@ available in a number of fixed sizes and are optimized for web use.
{% for type in site.data.logos.types %}
<div class="col-md-2">
<div class="row">
-<img style="height: 60px" src="{{ site.data.logos.logo-location }}/{{ color[0] }}/{{ type }}/beam-logo-{{ color[0] }}-{{ type }}.svg" alt="beam-logo-{{ color[0] }}-{{ type }}">
+<img style="height: 60px" src="{{ site.baseurl }}{{ site.data.logos.logo-location }}/{{ color[0] }}/{{ type }}/beam-logo-{{ color[0] }}-{{ type }}.svg" alt="beam-logo-{{ color[0] }}-{{ type }}">
</div><br>
<div class="row">
{% for size in site.data.logos.sizes %}
-<a href="{{ site.data.logos.logo-location }}/{{ color[0] }}/{{ type }}/beam-logo-{{ color[0] }}-{{ type }}-{{ size }}.png">{{ size }}x{{ size }}</a>
+<a href="{{ site.baseurl }}{{ site.data.logos.logo-location }}/{{ color[0] }}/{{ type }}/beam-logo-{{ color[0] }}-{{ type }}-{{ size }}.png">{{ size }}x{{ size }}</a>
{% unless forloop.last %},{% endunless %}
{% endfor %}
</div>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/contribute/release-guide.md
----------------------------------------------------------------------
diff --git a/contribute/release-guide.md b/contribute/release-guide.md
index fd98ca3..b0d4dca 100644
--- a/contribute/release-guide.md
+++ b/contribute/release-guide.md
@@ -15,7 +15,7 @@ The Apache Beam project periodically declares and publishes releases. A release
The Beam community treats releases with great importance. They are a public face of the project and most users interact with the project only through the releases. Releases are signed off by the entire Beam community in a public vote.
-Each release is executed by a *Release Manager*, who is selected among the [Beam committers](/project/team). This document describes the process that the Release Manager follows to perform a release. Any changes to this process should be discussed and adopted on the [dev@ mailing list](/use/mailing-lists).
+Each release is executed by a *Release Manager*, who is selected among the [Beam committers]({{ site.baseurl }}/project/team). This document describes the process that the Release Manager follows to perform a release. Any changes to this process should be discussed and adopted on the [dev@ mailing list]({{ site.baseurl }}/use/mailing-lists).
Please remember that publishing software has legal consequences. This guide complements the foundation-wide [Product Release Policy](http://www.apache.org/dev/release.html) and [Release Distribution Policy](http://www.apache.org/dev/release-distribution).
@@ -110,7 +110,7 @@ Configure access to the [Apache Nexus repository](http://repository.apache.org/)
#### Website development setup
-Get ready for updating the Beam website by following the [website development instructions](/contribute/contribution-guide/#website).
+Get ready for updating the Beam website by following the [website development instructions]({{ site.baseurl }}/contribute/contribution-guide/#website).
### Create a new version in JIRA
@@ -273,12 +273,12 @@ Please carefully review the generated Javadoc. Check for completeness and presen
The final step of building the candidate is to propose a website pull request.
-Add the new release to the [Apache Beam Releases](/use/releases/) page, as follows:
+Add the new release to the [Apache Beam Releases]({{ site.baseurl }}/use/releases/) page, as follows:
* Update the `<version>` tags in the sample `pom.xml` snippet to the new release.
* Add the new version to the `Release Notes` section below, along with links to the source code download and the Release Notes in JIRA.
-Add the new Javadoc to [SDK API Reference page](/learn/sdks/javadoc/) page, as follows:
+Add the new Javadoc to [SDK API Reference page]({{ site.baseurl }}/learn/sdks/javadoc/) page, as follows:
* Copy the generated Javadoc into the website repository: `cp -r ${JAVADOC_ROOT} learn/sdks/javadoc/${VERSION}`.
* Update the Javadoc link on this page to point to the new version.
@@ -289,7 +289,7 @@ Finally, propose a pull request with these changes. (Don\u2019t merge before finali
1. Maven artifacts deployed to the staging repository of [repository.apache.org](https://repository.apache.org/content/repositories/)
1. Source distribution deployed to the dev repository of [dist.apache.org](https://dist.apache.org/repos/dist/dev/incubator/beam/)
-1. Website pull request proposed to list the [release](/use/releases/) and publish the [API reference manual](/learn/sdks/javadoc/)
+1. Website pull request proposed to list the [release]({{ site.baseurl }}/use/releases/) and publish the [API reference manual]({{ site.baseurl }}/learn/sdks/javadoc/)
**********
@@ -442,7 +442,7 @@ Create a new Git tag for the released version by copying the tag for the final r
### Merge website pull request
-Merge the website pull request to [list the release](/use/releases/) and publish the [API reference manual](/learn/sdks/javadoc/) created earlier.
+Merge the website pull request to [list the release]({{ site.baseurl }}/use/releases/) and publish the [API reference manual]({{ site.baseurl }}/learn/sdks/javadoc/) created earlier.
### Mark the version as released in JIRA
@@ -452,7 +452,7 @@ In JIRA, inside [version management](https://issues.apache.org/jira/plugins/serv
* Maven artifacts released and indexed in the [Maven Central Repository](https://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.beam%22)
* Source distribution available in the release repository of [dist.apache.org](https://dist.apache.org/repos/dist/release/incubator/beam/)
-* Website pull request to [list the release](/use/releases/) and publish the [API reference manual](/learn/sdks/javadoc/) merged
+* Website pull request to [list the release]({{ site.baseurl }}/use/releases/) and publish the [API reference manual]({{ site.baseurl }}/learn/sdks/javadoc/) merged
* Release tagged in the source code repository
* Release version finalized in JIRA
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/contribute/team.md
----------------------------------------------------------------------
diff --git a/contribute/team.md b/contribute/team.md
index 835cd75..311a30a 100644
--- a/contribute/team.md
+++ b/contribute/team.md
@@ -2,8 +2,9 @@
layout: default
title: "Beam Team"
permalink: /contribute/team/
-redirect_from: /project/team/
-redirect_from: /team/
+redirect_from:
+ - /project/team/
+ - /team/
---
# Apache Beam Team
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/contribute/work-in-progress.md
----------------------------------------------------------------------
diff --git a/contribute/work-in-progress.md b/contribute/work-in-progress.md
index d1b6361..2017d57 100644
--- a/contribute/work-in-progress.md
+++ b/contribute/work-in-progress.md
@@ -6,7 +6,7 @@ permalink: /contribute/work-in-progress/
# Work In Progress in the Apache Beam Project
-As mentioned in the [Contribution Guide](/contribute/contribution-guide/), all work in Beam is tracked in our [Apache JIRA](https://issues.apache.org/jira/browse/BEAM). In addition, the following types of work may be of particular interest to the Beam community.
+As mentioned in the [Contribution Guide]({{ site.baseurl }}/contribute/contribution-guide/), all work in Beam is tracked in our [Apache JIRA](https://issues.apache.org/jira/browse/BEAM). In addition, the following types of work may be of particular interest to the Beam community.
* TOC
{:toc}
@@ -14,11 +14,11 @@ As mentioned in the [Contribution Guide](/contribute/contribution-guide/), all w
## Starter Tasks
-The community regular tags good getting started tasks with the label `starter`. Use a quick [JIRA search](https://issues.apache.org/jira/issues?jql=project%20%3D%20BEAM%20AND%20status%20%3D%20Open%20AND%20labels%20%3D%20starter) to identify ways you can get started [contributing](/contribute/contribution-guide/) to Beam.
+The community regular tags good getting started tasks with the label `starter`. Use a quick [JIRA search](https://issues.apache.org/jira/issues?jql=project%20%3D%20BEAM%20AND%20status%20%3D%20Open%20AND%20labels%20%3D%20starter) to identify ways you can get started [contributing]({{ site.baseurl }}/contribute/contribution-guide/) to Beam.
## Feature Branches
-Larger features with multiple active developers may be developed on a [feature branch](/contribute/contribution-guide/#feature-branches) before being merged in the master branch. In particular, this is often used for initial development of new components like SDKs or runners.
+Larger features with multiple active developers may be developed on a [feature branch]({{ site.baseurl }}/contribute/contribution-guide/#feature-branches) before being merged in the master branch. In particular, this is often used for initial development of new components like SDKs or runners.
Current branches include:
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/documentation/index.md
----------------------------------------------------------------------
diff --git a/documentation/index.md b/documentation/index.md
index fc5ef42..6bd6be8 100644
--- a/documentation/index.md
+++ b/documentation/index.md
@@ -2,8 +2,9 @@
layout: default
title: "Learn about Beam"
permalink: /documentation/
-redirect-from: /learn/
-redirect_from: /docs/learn/
+redirect_from:
+ - /learn/
+ - /docs/learn/
---
# Learn about the Apache Beam Model
@@ -26,4 +27,4 @@ Learn about the [Capability Matrix]({{ site.baseurl }}/learn/runners/capability-
* [Apache Spark]({{ site.baseurl }}/learn/runners/spark/)
* [Cloud Dataflow]({{ site.baseurl }}/learn/runners/dataflow/)
-#### [Additional Resources]({{ site.baseurl }}/learn/resources/)
\ No newline at end of file
+#### [Additional Resources]({{ site.baseurl }}/learn/resources/)
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/documentation/pipelines/create-your-pipeline.md
----------------------------------------------------------------------
diff --git a/documentation/pipelines/create-your-pipeline.md b/documentation/pipelines/create-your-pipeline.md
index e2c9ed0..b99628f 100644
--- a/documentation/pipelines/create-your-pipeline.md
+++ b/documentation/pipelines/create-your-pipeline.md
@@ -5,4 +5,4 @@ permalink: /documentation/pipelines/create-your-pipeline/
---
# Create Your Pipeline
-> **Note:** There is an open JIRA issue to create this guide ([BEAM-901](https://issues.apache.org/jira/browse/BEAM-901)).
\ No newline at end of file
+> **Note:** There is an open JIRA issue to create this guide ([BEAM-901](https://issues.apache.org/jira/browse/BEAM-901)).
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/documentation/pipelines/design-your-pipeline.md
----------------------------------------------------------------------
diff --git a/documentation/pipelines/design-your-pipeline.md b/documentation/pipelines/design-your-pipeline.md
index fca6912..8a76704 100644
--- a/documentation/pipelines/design-your-pipeline.md
+++ b/documentation/pipelines/design-your-pipeline.md
@@ -5,4 +5,4 @@ permalink: /documentation/pipelines/design-your-pipeline/
---
# Design Your Pipeline
-> **Note:** There is an open JIRA issue to create this guide ([BEAM-901](https://issues.apache.org/jira/browse/BEAM-901)).
\ No newline at end of file
+> **Note:** There is an open JIRA issue to create this guide ([BEAM-901](https://issues.apache.org/jira/browse/BEAM-901)).
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/documentation/pipelines/test-your-pipeline.md
----------------------------------------------------------------------
diff --git a/documentation/pipelines/test-your-pipeline.md b/documentation/pipelines/test-your-pipeline.md
index 82044c2..0a75bb5 100644
--- a/documentation/pipelines/test-your-pipeline.md
+++ b/documentation/pipelines/test-your-pipeline.md
@@ -5,4 +5,4 @@ permalink: /documentation/pipelines/test-your-pipeline/
---
# Test Your Pipeline
-> **Note:** There is an open JIRA issue to create this guide ([BEAM-901](https://issues.apache.org/jira/browse/BEAM-901)).
\ No newline at end of file
+> **Note:** There is an open JIRA issue to create this guide ([BEAM-901](https://issues.apache.org/jira/browse/BEAM-901)).
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/documentation/programming-guide.md
----------------------------------------------------------------------
diff --git a/documentation/programming-guide.md b/documentation/programming-guide.md
index 1094644..648962a 100644
--- a/documentation/programming-guide.md
+++ b/documentation/programming-guide.md
@@ -2,8 +2,9 @@
layout: default
title: "Beam Programming Guide"
permalink: /documentation/programming-guide/
-redirect-from: /learn/programming-guide/
-redirect_from: /docs/learn/programming-guide/
+redirect_from:
+ - /learn/programming-guide/
+ - /docs/learn/programming-guide/
---
# Apache Beam Programming Guide
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/documentation/resources.md
----------------------------------------------------------------------
diff --git a/documentation/resources.md b/documentation/resources.md
index cd7c6a7..5accec0 100644
--- a/documentation/resources.md
+++ b/documentation/resources.md
@@ -2,7 +2,7 @@
layout: default
title: "Beam Learning Resources"
permalink: /documentation/resources/
-redirect-from: /learn/resources/
+redirect_from: /learn/resources/
---
# Additional Resources for Learning about Apache Beam
@@ -13,4 +13,4 @@ Here's some links to some of our favorite articles and videos get you started.
* [The world beyond batch: Streaming 101](https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-101)
* [The world beyong batch: Streaming 102](https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-102)
* [Dataflow/Beam & Spark: A Programming Model Comparison](https://cloud.google.com/dataflow/blog/dataflow-beam-and-spark-comparison)
-* [Dataflow and open source - proposal to join the Apache Incubator](http://googlecloudplatform.blogspot.com/2016/01/Dataflow-and-open-source-proposal-to-join-the-Apache-Incubator.html)
\ No newline at end of file
+* [Dataflow and open source - proposal to join the Apache Incubator](http://googlecloudplatform.blogspot.com/2016/01/Dataflow-and-open-source-proposal-to-join-the-Apache-Incubator.html)
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/documentation/runners/capability-materix.md
----------------------------------------------------------------------
diff --git a/documentation/runners/capability-materix.md b/documentation/runners/capability-materix.md
index 89ce555..d7af766 100644
--- a/documentation/runners/capability-materix.md
+++ b/documentation/runners/capability-materix.md
@@ -2,8 +2,9 @@
layout: default
title: "Apache Beam Capability Matrix"
permalink: /documentation/runners/capability-matrix/
-redirect-from: /learn/runners/capability-matrix/
-redirect-from: /capability-matrix/
+redirect_from:
+ - /learn/runners/capability-matrix/
+ - /capability-matrix/
---
# Beam Capability Matrix
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/documentation/runners/dataflow.md
----------------------------------------------------------------------
diff --git a/documentation/runners/dataflow.md b/documentation/runners/dataflow.md
index 375d289..c49223b 100644
--- a/documentation/runners/dataflow.md
+++ b/documentation/runners/dataflow.md
@@ -2,7 +2,7 @@
layout: default
title: "Cloud Dataflow Runner"
permalink: /documentation/runners/dataflow/
-redirect-from: /learn/runners/dataflow/
+redirect_from: /learn/runners/dataflow/
---
# Using the Cloud Dataflow Runner
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/documentation/runners/direct.md
----------------------------------------------------------------------
diff --git a/documentation/runners/direct.md b/documentation/runners/direct.md
index ef31f04..094d44e 100644
--- a/documentation/runners/direct.md
+++ b/documentation/runners/direct.md
@@ -2,7 +2,7 @@
layout: default
title: "Apache Direct Runner"
permalink: /documentation/runners/direct/
-redirect-from: /learn/runners/direct/
+redirect_from: /learn/runners/direct/
---
# Using the Direct Runner
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/documentation/runners/flink.md
----------------------------------------------------------------------
diff --git a/documentation/runners/flink.md b/documentation/runners/flink.md
index 8554b98..4145be6 100644
--- a/documentation/runners/flink.md
+++ b/documentation/runners/flink.md
@@ -2,7 +2,7 @@
layout: default
title: "Apache Flink Runner"
permalink: /documentation/runners/flink/
-redirect-from: /learn/runners/flink/
+redirect_from: /learn/runners/flink/
---
# Using the Apache Flink Runner
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/documentation/runners/index.md
----------------------------------------------------------------------
diff --git a/documentation/runners/index.md b/documentation/runners/index.md
index d36a6ab..d6d2211 100644
--- a/documentation/runners/index.md
+++ b/documentation/runners/index.md
@@ -2,7 +2,7 @@
layout: default
title: "Beam Runners"
permalink: /documentation/runners/
-redirect-from: /learn/runners/
+redirect_from: /learn/runners/
---
# Apache Beam Runners
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/documentation/runners/spark.md
----------------------------------------------------------------------
diff --git a/documentation/runners/spark.md b/documentation/runners/spark.md
index 2650f8f..66d01db 100644
--- a/documentation/runners/spark.md
+++ b/documentation/runners/spark.md
@@ -2,7 +2,7 @@
layout: default
title: "Apache Spark Runner"
permalink: /documentation/runners/spark/
-redirect-from: /learn/runners/spark/
+redirect_from: /learn/runners/spark/
---
# Using the Apache Spark Runner
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/documentation/sdks/index.md
----------------------------------------------------------------------
diff --git a/documentation/sdks/index.md b/documentation/sdks/index.md
index 0cabb5a..1a5a08e 100644
--- a/documentation/sdks/index.md
+++ b/documentation/sdks/index.md
@@ -2,7 +2,7 @@
layout: default
title: "Beam SDKs"
permalink: /documentation/sdks/
-redirect-from: /learn/sdks/
+redirect_from: /learn/sdks/
---
# Apache Beam SDKs
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/documentation/sdks/java.md
----------------------------------------------------------------------
diff --git a/documentation/sdks/java.md b/documentation/sdks/java.md
index 5968d59..91ba7b8 100644
--- a/documentation/sdks/java.md
+++ b/documentation/sdks/java.md
@@ -2,12 +2,12 @@
layout: default
title: "Beam Java SDK"
permalink: /documentation/sdks/java/
-redirect-from: /learn/sdks/java/
+redirect_from: /learn/sdks/java/
---
# Apache Beam Java SDK
This page is under construction ([BEAM-504](https://issues.apache.org/jira/browse/BEAM-504)).
-Get started with the [Beam Programming Guide](/learn/programming-guide) to learn the basic concepts that hold for all SDKs in the Beam Model.
+Get started with the [Beam Programming Guide]({{ site.baseurl }}/learn/programming-guide) to learn the basic concepts that hold for all SDKs in the Beam Model.
-See the [Java API Reference](/learn/sdks/javadoc/).
+See the [Java API Reference]({{ site.baseurl }}/learn/sdks/javadoc/).
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/documentation/sdks/javadoc/index.md
----------------------------------------------------------------------
diff --git a/documentation/sdks/javadoc/index.md b/documentation/sdks/javadoc/index.md
index 17872c0..cfaa022 100644
--- a/documentation/sdks/javadoc/index.md
+++ b/documentation/sdks/javadoc/index.md
@@ -2,7 +2,7 @@
layout: default
title: "Beam Javadoc"
permalink: /documentation/sdks/javadoc/
-redirect-from: /learn/sdks/javadoc/
+redirect_from: /learn/sdks/javadoc/
---
# Apache Beam Java SDK API Reference
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/get-started/beam-overview.md
----------------------------------------------------------------------
diff --git a/get-started/beam-overview.md b/get-started/beam-overview.md
index 7ca9709..1b43ca8 100644
--- a/get-started/beam-overview.md
+++ b/get-started/beam-overview.md
@@ -2,8 +2,9 @@
layout: default
title: "Beam Overview"
permalink: /get-started/beam-overview/
-redirect_from: /use/beam-overview/
-redirect_from: /docs/use/beam-overview/
+redirect_from:
+ - /use/beam-overview/
+ - /docs/use/beam-overview/
---
# Apache Beam Overview
@@ -61,4 +62,4 @@ Beam currently supports Runners that work with the following distributed process
</tr>
</table>
-**Note:** You can always execute your pipeline locally for testing and debugging purposes.
\ No newline at end of file
+**Note:** You can always execute your pipeline locally for testing and debugging purposes.
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/get-started/downloads.md
----------------------------------------------------------------------
diff --git a/get-started/downloads.md b/get-started/downloads.md
index 3f1927b..95bbffc 100644
--- a/get-started/downloads.md
+++ b/get-started/downloads.md
@@ -6,4 +6,4 @@ permalink: /get-started/downloads/
# Downloads
-> **Note:** There is an open JIRA issue to fill in this page ([BEAM-906](https://issues.apache.org/jira/browse/BEAM-906)).
\ No newline at end of file
+> **Note:** There is an open JIRA issue to fill in this page ([BEAM-906](https://issues.apache.org/jira/browse/BEAM-906)).
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/get-started/index.md
----------------------------------------------------------------------
diff --git a/get-started/index.md b/get-started/index.md
index 72d68b4..4ab55af 100644
--- a/get-started/index.md
+++ b/get-started/index.md
@@ -2,8 +2,9 @@
layout: default
title: "Use Beam"
permalink: /get-started/
-redirect_from: /use/
-redirect_from: /docs/use/
+redirect_from:
+ - /use/
+ - /docs/use/
---
# Using Apache Beam
@@ -22,7 +23,7 @@ Find information on the latest Beam releases, including versioning and release n
See detailed walkthroughs of complete Beam pipelines.
* [WordCount]({{ site.baseurl }}/use/wordcount-example): Simple example pipelines that demonstrate basic Beam programming, including debugging and testing
-* [Mobile Gaming]({{ site.baseurl }}/use/walkthroughs): A series of more advanced pipelines that demonstrate use cases in the mobile gaming domain
+* [Mobile Gaming]({{ site.baseurl }}/use/mobile-gaming-example): A series of more advanced pipelines that demonstrate use cases in the mobile gaming domain
#### Support
Find resources to help you use Beam.
@@ -30,4 +31,4 @@ Find resources to help you use Beam.
* [Mailing Lists]({{ site.baseurl }}/use/mailing-lists/)
* [Issue Tracking]({{ site.baseurl }}/use/issue-tracking/)
* [Beam on Stack Overflow](http://stackoverflow.com/questions/tagged/apache-beam)
-* [Slack Channel](http://apachebeam.slack.com)
\ No newline at end of file
+* [Slack Channel](http://apachebeam.slack.com)
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/get-started/mobile-gaming-example.md
----------------------------------------------------------------------
diff --git a/get-started/mobile-gaming-example.md b/get-started/mobile-gaming-example.md
index 1a5b2ca..b51f81a 100644
--- a/get-started/mobile-gaming-example.md
+++ b/get-started/mobile-gaming-example.md
@@ -10,7 +10,7 @@ redirect_from: /use/mobile-gaming-example/
* TOC
{:toc}
-This section provides a walkthrough of a series of example Apache Beam pipelines that demonstrate more complex functionality than the basic [WordCount](/use/wordcount-example) examples. The pipelines in this section process data from a hypothetical game that users play on their mobile phones. The pipelines demonstrate processing at increasing levels of complexity; the first pipeline, for example, shows how to run a batch analysis job to obtain relatively simple score data, while the later pipelines use Beam's windowing and triggers features to provide low-latency data analysis and more complex intelligence about user's play patterns.
+This section provides a walkthrough of a series of example Apache Beam pipelines that demonstrate more complex functionality than the basic [WordCount]({{ site.baseurl }}/use/wordcount-example) examples. The pipelines in this section process data from a hypothetical game that users play on their mobile phones. The pipelines demonstrate processing at increasing levels of complexity; the first pipeline, for example, shows how to run a batch analysis job to obtain relatively simple score data, while the later pipelines use Beam's windowing and triggers features to provide low-latency data analysis and more complex intelligence about user's play patterns.
> **Note**: These examples assume some familiarity with the Beam programming model. If you haven't already, we recommend familiarizing yourself with the programming model documentation and running a basic example pipeline before continuing. Note also that these examples use the Java 8 lambda syntax, and thus require Java 8. However, you can create pipelines with equivalent functionality using Java 7.
@@ -27,7 +27,7 @@ This means that some data events might be received by the game server significan
Because some of our example pipelines use data files (like logs from the game server) as input, the event timestamp for each game might be embedded in the data--that is, it's a field in each data record. Those pipelines need to parse the event timestamp from each data record after reading it from the input file.
-For pipelines that read unbounded game data from an unbounded source, the data source sets the intrinsic [timestamp](/learn/programming-guide/#pctimestamps) for each PCollection element to the appropriate event time.
+For pipelines that read unbounded game data from an unbounded source, the data source sets the intrinsic [timestamp]({{ site.baseurl }}/documentation/programming-guide/#pctimestamps) for each PCollection element to the appropriate event time.
The Mobile Game example pipelines vary in complexity, from simple batch analysis to more complex pipelines that can perform real-time analysis and abuse detection. This section walks you through each example and demonstrates how to use Beam features like windowing and triggers to expand your pipeline's capabilites.
@@ -56,7 +56,7 @@ As the pipeline processes each event, the event score gets added to the sum tota
The following diagram shows score data for a several users over the pipeline analysis period. In the diagram, each data point is an event that results in one user/score pair:
<figure id="fig1">
- <img src="/images/gaming-example.gif"
+ <img src="{{ site.baseurl }}/images/gaming-example.gif"
width="900" height="263"
alt="Score data for three users.">
</figure>
@@ -64,7 +64,7 @@ Figure 1: Score data for three users.
This example uses batch processing, and the diagram's Y axis represents processing time: the pipeline processes events lower on the Y-axis first, and events higher up the axis later. The diagram's X axis represents the event time for each game event, as denoted by that event's timestamp. Note that the individual events in the diagram are not processed by the pipeline in the same order as they occurred (according to their timestamps).
-After reading the score events from the input file, the pipeline groups all of those user/score pairs together and sums the score values into one total value per unique user. `UserScore` encapsulates the core logic for that step as the [user-defined composite transform](/learn/programming-guide/#transforms-composite) `ExtractAndSumScore`:
+After reading the score events from the input file, the pipeline groups all of those user/score pairs together and sums the score values into one total value per unique user. `UserScore` encapsulates the core logic for that step as the [user-defined composite transform]({{ site.baseurl }}/documentation/programming-guide/#transforms-composite) `ExtractAndSumScore`:
```java
public static class ExtractAndSumScore
@@ -163,7 +163,7 @@ Using fixed-time windowing lets the pipeline provide better information on how e
The following diagram shows how the pipeline processes a day's worth of a single team's scoring data after applying fixed-time windowing:
<figure id="fig2">
- <img src="/images/gaming-example-team-scores-narrow.gif"
+ <img src="{{ site.baseurl }}/images/gaming-example-team-scores-narrow.gif"
width="900" height="390"
alt="Score data for two teams.">
</figure>
@@ -173,7 +173,7 @@ Notice that as processing time advances, the sums are now _per window_; each win
> **Note:** As is shown in the diagram above, using windowing produces an _independent total for every interval_ (in this case, each hour). `HourlyTeamScore` doesn't provide a running total for the entire data set at each hour--it provides the total score for all the events that occurred _only within that hour_.
-Beam's windowing feature uses the [intrinsic timestamp information](/learn/programming-guide/#pctimestamps) attached to each element of a `PCollection`. Because we want our pipeline to window based on _event time_, we **must first extract the timestamp** that's embedded in each data record apply it to the corresponding element in the `PCollection` of score data. Then, the pipeline can **apply the windowing function** to divide the `PCollection` into logical windows.
+Beam's windowing feature uses the [intrinsic timestamp information]({{ site.baseurl }}/documentation/programming-guide/#pctimestamps) attached to each element of a `PCollection`. Because we want our pipeline to window based on _event time_, we **must first extract the timestamp** that's embedded in each data record apply it to the corresponding element in the `PCollection` of score data. Then, the pipeline can **apply the windowing function** to divide the `PCollection` into logical windows.
Here's the code, which shows how `HourlyTeamScore` uses the [WithTimestamps](https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/WithTimestamps.java) and [Window](https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/windowing/Window.java) transforms to perform these operations:
@@ -290,12 +290,12 @@ Below, we'll look at both of these tasks in detail.
We want our pipeline to output a running total score for each user for every ten minutes that the pipeline runs. This calculation doesn't consider _when_ the actual score was generated by the user's play instance; it simply outputs the sum of all the scores for that user that have arrived in the pipeline to date. Late data gets included in the calculation whenever it happens to arrive in the pipeline as it's running.
-Because we want all the data that has arrived in the pipeline every time we update our calculation, we have the pipeline consider all of the user score data in a **single global window**. The single global window is unbounded, but we can specify a kind of temporary cut-off point for each ten-minute calculation by using a processing time [trigger](/learn/programming-guide/#triggers).
+Because we want all the data that has arrived in the pipeline every time we update our calculation, we have the pipeline consider all of the user score data in a **single global window**. The single global window is unbounded, but we can specify a kind of temporary cut-off point for each ten-minute calculation by using a processing time [trigger]({{ site.baseurl }}/documentation/programming-guide/#triggers).
When we specify a ten-minute processing time trigger for the single global window, the effect is that the pipeline effectively takes a "snapshot" of the contents of the window every time the trigger fires (which it does at ten-minute intervals). Since we're using a single global window, each snapshot contains all the data collected _to that point in time_. The following diagram shows the effects of using a processing time trigger on the single global window:
<figure id="fig3">
- <img src="/images/gaming-example-proc-time-narrow.gif"
+ <img src="{{ site.baseurl }}/images/gaming-example-proc-time-narrow.gif"
width="900" height="263"
alt="Score data for for three users.">
</figure>
@@ -342,14 +342,14 @@ We want our pipeline to also output the total score for each team during each ho
Because we consider each hour individually, we can apply fixed-time windowing to our input data, just like in `HourlyTeamScore`. To provide the speculative updates and updates on late data, we'll specify additional trigger parameters. The trigger will cause each window to calculate and emit results at an interval we specify (in this case, every five minutes), and also to keep triggering after the window is considered "complete" to account for late data. Just like the user score calculation, we'll set the trigger to accumulating mode to ensure that we get a running sum for each hour-long window.
-The triggers for speculative updates and late data help with the problem of [time skew](/learn/programming-guide/#windowing). Events in the pipeline aren't necessarily processed in the order in which they actually occurred according to their timestamps; they may arrive in the pipeline out of order, or late (in our case, because they were generated while the user's phone was out of contact with a network). Beam needs a way to determine when it can reasonably assume that it has "all" of the data in a given window: this is called the _watermark_.
+The triggers for speculative updates and late data help with the problem of [time skew]({{ site.baseurl }}/documentation/programming-guide/#windowing). Events in the pipeline aren't necessarily processed in the order in which they actually occurred according to their timestamps; they may arrive in the pipeline out of order, or late (in our case, because they were generated while the user's phone was out of contact with a network). Beam needs a way to determine when it can reasonably assume that it has "all" of the data in a given window: this is called the _watermark_.
In an ideal world, all data would be processed immediately when it occurs, so the processing time would be equal to (or at least have a linear relationship to) the event time. However, because distributed systems contain some inherent inaccuracy (like our late-reporting phones), Beam often uses a heuristic watermark.
The following diagram shows the relationship between ongoing processing time and each score's event time for two teams:
<figure id="fig4">
- <img src="/images/gaming-example-event-time-narrow.gif"
+ <img src="{{ site.baseurl }}/images/gaming-example-event-time-narrow.gif"
width="900" height="390"
alt="Score data by team, windowed by event time.">
</figure>
@@ -417,7 +417,7 @@ Let's suppose scoring in our game depends on the speed at which a user can "clic
To determine whether or not a score is "abnormally" high, `GameStats` calculates the average of every score in that fixed-time window, and then checks each score individual score against the average score multiplied by an arbitrary weight factor (in our case, 2.5). Thus, any score more than 2.5 times the average is deemed to be the product of spam. The `GameStats` pipeline tracks a list of "spam" users and filters those users out of the team score calculations for the team leader board.
-Since the average depends on the pipeline data, we need to calculate it, and then use that calculated data in a subsequent `ParDo` transform that filters scores that exceed the weighted value. To do this, we can pass the calculated average to as a [side input](/learn/programming-guide/#transforms-sideio) to the filtering `ParDo`.
+Since the average depends on the pipeline data, we need to calculate it, and then use that calculated data in a subsequent `ParDo` transform that filters scores that exceed the weighted value. To do this, we can pass the calculated average to as a [side input]({{ site.baseurl }}/documentation/programming-guide/#transforms-sideio) to the filtering `ParDo`.
The following code example shows the composite transform that handles abuse detection. The transform uses the `Sum.integersPerKey` transform to sum all scores per user, and then the `Mean.globally` transform to determine the average score for all users. Once that's been calculated (as a `PCollectionView` singleton), we can pass it to the filtering `ParDo` using `.withSideInputs`:
@@ -498,7 +498,7 @@ When you set session windowing, you specify a _minimum gap duration_ between eve
The following diagram shows how data might look when grouped into session windows. Unlike fixed windows, session windows are _different for each user_ and is dependent on each individual user's play pattern:
<figure id="fig5">
- <img src="/images/gaming-example-session-windows.png"
+ <img src="{{ site.baseurl }}/images/gaming-example-session-windows.png"
width="662" height="521"
alt="A diagram representing session windowing."
alt="User sessions, with a minimum gap duration.">
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/get-started/quickstart.md
----------------------------------------------------------------------
diff --git a/get-started/quickstart.md b/get-started/quickstart.md
index a1fee07..f0cd5c3 100644
--- a/get-started/quickstart.md
+++ b/get-started/quickstart.md
@@ -2,8 +2,9 @@
layout: default
title: "Beam Quickstart"
permalink: /get-started/quickstart/
-redirect_from: /use/quickstart/
-redirect_from: /getting-started/
+redirect_from:
+ - /use/quickstart/
+ - /getting-started/
---
# Apache Beam Quickstart
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/get-started/releases.md
----------------------------------------------------------------------
diff --git a/get-started/releases.md b/get-started/releases.md
index a04a666..4241b0d 100644
--- a/get-started/releases.md
+++ b/get-started/releases.md
@@ -2,8 +2,9 @@
layout: default
title: "Beam Releases"
permalink: get-started/releases/
-redirect_from: /use/releases/
-redirect_from: /releases/
+redirect_from:
+ - /use/releases/
+ - /releases/
---
# Apache Beam Releases
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/get-started/support.md
----------------------------------------------------------------------
diff --git a/get-started/support.md b/get-started/support.md
index ea5e41d..cfb5212 100644
--- a/get-started/support.md
+++ b/get-started/support.md
@@ -2,6 +2,9 @@
layout: default
title: "Support"
permalink: /get-started/support/
+redirect_from:
+ - /use/issue-tracking/
+ - /use/mailing-lists/
---
# Support
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/get-started/wordcount-example.md
----------------------------------------------------------------------
diff --git a/get-started/wordcount-example.md b/get-started/wordcount-example.md
index 851bdbf..4df1cd1 100644
--- a/get-started/wordcount-example.md
+++ b/get-started/wordcount-example.md
@@ -85,7 +85,7 @@ The Minimal WordCount pipeline contains five transforms:
p.apply(TextIO.Read.from("gs://apache-beam-samples/shakespeare/*"))
```
-2. A [ParDo](/learn/programming-guide/#transforms-pardo) transform that invokes a `DoFn` (defined in-line as an anonymous class) on each element that tokenizes the text lines into individual words. The input for this transform is the `PCollection` of text lines generated by the previous `TextIO.Read` transform. The `ParDo` transform outputs a new `PCollection`, where each element represents an individual word in the text.
+2. A [ParDo]({{ site.baseurl }}/documentation/programming-guide/#transforms-pardo) transform that invokes a `DoFn` (defined in-line as an anonymous class) on each element that tokenizes the text lines into individual words. The input for this transform is the `PCollection` of text lines generated by the previous `TextIO.Read` transform. The `ParDo` transform outputs a new `PCollection`, where each element represents an individual word in the text.
```java
.apply("ExtractWords", ParDo.of(new DoFn<String, String>() {
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/a7c71d23/index.md
----------------------------------------------------------------------
diff --git a/index.md b/index.md
index 5a736b4..31a8a09 100644
--- a/index.md
+++ b/index.md
@@ -4,7 +4,7 @@ layout: default
<div class="alert alert-info alert-dismissible" role="alert">
<span class="glyphicon glyphicon-flag" aria-hidden="true"></span>
<button type="button" class="close" data-dismiss="alert" aria-label="Close"><span aria-hidden="true">×</span></button>
-The Apache Beam project is in the process of bootstrapping. This includes the website -- so please file issues you find in <a href="/use/issue-tracking/">Jira</a>. Thanks!
+The Apache Beam project is in the process of bootstrapping. This includes the website -- so please file issues you find in <a href="{{ site.baseurl }}/use/issue-tracking/">Jira</a>. Thanks!
</div>
# Apache Beam (incubating)
@@ -69,8 +69,8 @@ Beam currently supports Runners that work with the following distributed process
Interested in working with Apache Beam? Great! Here's how to get started:
-* If you are interested in using Beam for your data processing tasks, use the [Get Started](/get-started/) section for an overview, quickstart, and examples. Then dive into the [Documentation](/documentatiion/) section to get up to speed on the details.
-* If you're interested in contributing to the Beam SDKs or Runners, start with the [Contribute](/contribute/) section.
+* If you are interested in using Beam for your data processing tasks, use the [Get Started]({{ site.baseurl }}/get-started/) section for an overview, quickstart, and examples. Then dive into the [Documentation]({{ site.baseurl }}/documentation/) section to get up to speed on the details.
+* If you're interested in contributing to the Beam SDKs or Runners, start with the [Contribute]({{ site.baseurl }}/contribute/) section.
<hr>
<div class="row">
[2/4] incubator-beam-site git commit: Build in a separate content
directory to catch bad links
Posted by da...@apache.org.
Build in a separate content directory to catch bad links
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/e3eb8913
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/tree/e3eb8913
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/diff/e3eb8913
Branch: refs/heads/asf-site
Commit: e3eb891397122811db0bd7eb76701ade42eef0d1
Parents: a7c71d2
Author: Dan Halperin <dh...@google.com>
Authored: Fri Nov 4 13:30:48 2016 -0700
Committer: Dan Halperin <dh...@google.com>
Committed: Fri Nov 4 17:46:01 2016 -0700
----------------------------------------------------------------------
Rakefile | 4 +++-
_config_test.yml | 5 +++++
2 files changed, 8 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/e3eb8913/Rakefile
----------------------------------------------------------------------
diff --git a/Rakefile b/Rakefile
index b25b1fb..886442c 100644
--- a/Rakefile
+++ b/Rakefile
@@ -1,7 +1,9 @@
+require 'fileutils'
require 'html-proofer'
task :test do
- sh "bundle exec jekyll build"
+ FileUtils.rm_rf('./content')
+ sh "bundle exec jekyll build --config _config.yml,_config_test.yml"
HTMLProofer.check_directory("./content", {
:allow_hash_href => true,
:check_html => true,
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/e3eb8913/_config_test.yml
----------------------------------------------------------------------
diff --git a/_config_test.yml b/_config_test.yml
new file mode 100644
index 0000000..607f568
--- /dev/null
+++ b/_config_test.yml
@@ -0,0 +1,5 @@
+# This config file contains options that override the default config for testing.
+
+url: ""
+baseurl: /subdir
+destination: content/subdir
[3/4] incubator-beam-site git commit: Regenerate website
Posted by da...@apache.org.
Regenerate website
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/d384fc51
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/tree/d384fc51
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/diff/d384fc51
Branch: refs/heads/asf-site
Commit: d384fc51b36fd8b9d7b998ac283a33766be6033f
Parents: e3eb891
Author: Davor Bonaci <da...@google.com>
Authored: Fri Nov 4 17:51:28 2016 -0700
Committer: Davor Bonaci <da...@google.com>
Committed: Fri Nov 4 17:51:28 2016 -0700
----------------------------------------------------------------------
.../10/11/strata-hadoop-world-and-beam.html | 2 +-
content/blog/2016/08/03/six-months.html | 2 +-
content/blog/2016/10/20/test-stream.html | 6 ++--
content/capability-matrix/index.html | 10 ++++++
.../contribute/contribution-guide/index.html | 4 +--
.../runners/capability-matrix/index.html | 36 ++++++++++----------
content/feed.xml | 14 ++++----
content/get-started/index.html | 2 +-
.../mobile-gaming-example/index.html | 12 +++----
.../get-started/wordcount-example/index.html | 2 +-
content/index.html | 2 +-
content/learn/index.html | 10 ++++++
content/learn/programming-guide/index.html | 10 ++++++
content/learn/resources/index.html | 10 ++++++
.../learn/runners/capability-matrix/index.html | 10 ++++++
content/learn/runners/dataflow/index.html | 10 ++++++
content/learn/runners/direct/index.html | 10 ++++++
content/learn/runners/flink/index.html | 10 ++++++
content/learn/runners/index.html | 10 ++++++
content/learn/runners/spark/index.html | 10 ++++++
content/learn/sdks/index.html | 10 ++++++
content/learn/sdks/java/index.html | 10 ++++++
content/learn/sdks/javadoc/index.html | 10 ++++++
content/project/logos/index.html | 10 ++++++
content/project/team/index.html | 10 ++++++
content/use/beam-overview/index.html | 10 ++++++
content/use/index.html | 10 ++++++
content/use/issue-tracking/index.html | 10 ++++++
content/use/mailing-lists/index.html | 10 ++++++
content/use/quickstart/index.html | 10 ++++++
content/use/releases/index.html | 10 ++++++
31 files changed, 251 insertions(+), 41 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/beam/update/2016/10/11/strata-hadoop-world-and-beam.html
----------------------------------------------------------------------
diff --git a/content/beam/update/2016/10/11/strata-hadoop-world-and-beam.html b/content/beam/update/2016/10/11/strata-hadoop-world-and-beam.html
index 280b3f5..630c3bc 100644
--- a/content/beam/update/2016/10/11/strata-hadoop-world-and-beam.html
+++ b/content/beam/update/2016/10/11/strata-hadoop-world-and-beam.html
@@ -169,7 +169,7 @@
<p>We rounded out the tutorial with live demonstrations of Beam running on Apache Spark, Apache Flink, the local runner, and DataFlow runner. Then, we brought in the big brainpower and had a Q and A session.</p>
-<p>If you\u2019re attending a conference, we encourage you to look for a Beam session. If you want to use these materials to give your own Beam talk or tutorial, we\u2019re happy to help you. In addition to this tutorial, we have <a href="/learn/presentation-materials/">other presentation materials</a>. You can reach out to us on the <a href="http://beam.incubator.apache.org/use/mailing-lists/">user mailing list</a>.</p>
+<p>If you\u2019re attending a conference, we encourage you to look for a Beam session. If you want to use these materials to give your own Beam talk or tutorial, we\u2019re happy to help you. In addition to this tutorial, we have <a href="/learn/presentation-materials/">other presentation materials</a>. You can reach out to us on the <a href="/use/mailing-lists/">user mailing list</a>.</p>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/blog/2016/08/03/six-months.html
----------------------------------------------------------------------
diff --git a/content/blog/2016/08/03/six-months.html b/content/blog/2016/08/03/six-months.html
index ca17065..25da448 100644
--- a/content/blog/2016/08/03/six-months.html
+++ b/content/blog/2016/08/03/six-months.html
@@ -186,7 +186,7 @@
<li>Presented at multiple existing meetups and are starting to organize some of our own.</li>
</ul>
-<p>While it\u2019s nice to reflect back on all we\u2019ve done, we\u2019re working full <em>stream</em> ahead towards a stable release and graduation from incubator. And we\u2019d love your help \u2013 join the <a href="http://beam.incubator.apache.org/use/mailing-lists/">mailing lists</a>, check out the <a href="http://beam.incubator.apache.org/contribute/contribution-guide/">contribution guide</a>, and grab a <a href="https://issues.apache.org/jira/browse/BEAM-520?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20in%20(newbie%2C%20starter)">starter task</a> from Jira!</p>
+<p>While it\u2019s nice to reflect back on all we\u2019ve done, we\u2019re working full <em>stream</em> ahead towards a stable release and graduation from incubator. And we\u2019d love your help \u2013 join the <a href="/use/mailing-lists/">mailing lists</a>, check out the <a href="/contribute/contribution-guide/">contribution guide</a>, and grab a <a href="https://issues.apache.org/jira/browse/BEAM-520?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20in%20(newbie%2C%20starter)">starter task</a> from Jira!</p>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/blog/2016/10/20/test-stream.html
----------------------------------------------------------------------
diff --git a/content/blog/2016/10/20/test-stream.html b/content/blog/2016/10/20/test-stream.html
index ddd114e..176c96b 100644
--- a/content/blog/2016/10/20/test-stream.html
+++ b/content/blog/2016/10/20/test-stream.html
@@ -188,7 +188,7 @@ from the Mobile Gaming example series.</p>
<p><a href="https://github.com/apache/incubator-beam/blob/master/examples/java8/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java#L177">LeaderBoard</a>
is part of the <a href="https://github.com/apache/incubator-beam/tree/master/examples/java8/src/main/java/org/apache/beam/examples/complete/game">Beam mobile gaming examples</a>
-(and <a href="http://beam.incubator.apache.org/use/walkthroughs/">walkthroughs</a>)
+(and <a href="/get-started/mobile-gaming-example/">walkthroughs</a>)
which produces a continuous accounting of user and team scores. User scores are
calculated over the lifetime of the program, while team scores are calculated
within fixed windows with a default duration of one hour. The LeaderBoard
@@ -201,7 +201,7 @@ be controlled within a test.</p>
<h2 id="writing-deterministic-tests-to-emulate-nondeterminism">Writing Deterministic Tests to Emulate Nondeterminism</h2>
<p>The Beam testing infrastructure provides the
-<a href="http://beam.incubator.apache.org/learn/sdks/javadoc/0.2.0-incubating/">PAssert</a>
+<a href="/documentation/sdks/javadoc/0.2.0-incubating/">PAssert</a>
methods, which assert properties about the contents of a PCollection from within
a pipeline. We have expanded this infrastructure to include
<a href="https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/TestStream.java">TestStream</a>,
@@ -449,7 +449,7 @@ TestStream.</p>
has enabled the testing of Pipelines which produce speculative and late panes.
This permits tests for all styles of pipeline to be expressed directly within the
Java SDK. If you have questions or comments, we\u2019d love to hear them on the
-<a href="http://beam.incubator.apache.org/use/mailing-lists/">mailing lists</a>.</p>
+<a href="/use/mailing-lists/">mailing lists</a>.</p>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/capability-matrix/index.html
----------------------------------------------------------------------
diff --git a/content/capability-matrix/index.html b/content/capability-matrix/index.html
new file mode 100644
index 0000000..48f5824
--- /dev/null
+++ b/content/capability-matrix/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/documentation/runners/capability-matrix/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/documentation/runners/capability-matrix/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/documentation/runners/capability-matrix/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/documentation/runners/capability-matrix/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/contribute/contribution-guide/index.html
----------------------------------------------------------------------
diff --git a/content/contribute/contribution-guide/index.html b/content/contribute/contribution-guide/index.html
index 560bd4d..be052ac 100644
--- a/content/contribute/contribution-guide/index.html
+++ b/content/contribute/contribution-guide/index.html
@@ -209,7 +209,7 @@
<p><strong>For larger contributions or those that affect multiple components:</strong></p>
<ol>
- <li><strong>Engage</strong>: We encourage you to work with the Beam community on the <a href="https://issues.apache.org/jira/browse/BEAM">Apache JIRA issue tracker</a> and <a href="http://beam.incubator.apache.org/use/mailing-lists/">developer\u2019s mailing list</a> to identify good areas for contribution.</li>
+ <li><strong>Engage</strong>: We encourage you to work with the Beam community on the <a href="https://issues.apache.org/jira/browse/BEAM">Apache JIRA issue tracker</a> and <a href="/use/mailing-lists/">developer\u2019s mailing list</a> to identify good areas for contribution.</li>
<li><strong>Design:</strong> More complicated contributions will likely benefit from some early discussion in order to scope and design them well.</li>
</ol>
@@ -228,7 +228,7 @@
<h3 id="mailing-lists">Mailing list(s)</h3>
<p>We discuss design and implementation issues on dev@beam.incubator.apache.org mailing list, which is archived <a href="https://lists.apache.org/list.html?dev@beam.apache.org">here</a>. Join by emailing <a href="mailto:dev-subscribe@beam.incubator.apache.org"><code class="highlighter-rouge">dev-subscribe@beam.incubator.apache.org</code></a>.</p>
-<p>If interested, you can also join the other <a href="http://beam.incubator.apache.org/use/mailing-lists/">mailing lists</a> too.</p>
+<p>If interested, you can also join the other <a href="/use/mailing-lists/">mailing lists</a> too.</p>
<h3 id="apache-jira">Apache JIRA</h3>
<p>We use <a href="https://issues.apache.org/jira/browse/BEAM">Apache JIRA</a> as an issue tracking and project management tool, as well as a way to communicate among a very diverse and distributed set of contributors. To be able to gather feedback, avoid frustration, and avoid duplicated efforts all Beam-related work should be tracked there.</p>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/documentation/runners/capability-matrix/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/runners/capability-matrix/index.html b/content/documentation/runners/capability-matrix/index.html
index 8a33334..8db9915 100644
--- a/content/documentation/runners/capability-matrix/index.html
+++ b/content/documentation/runners/capability-matrix/index.html
@@ -140,7 +140,7 @@
<div class="row">
<h1 id="beam-capability-matrix">Beam Capability Matrix</h1>
-<p><span style="font-size:11px;float:none">Last updated: 2016-11-04 16:45 PDT</span></p>
+<p><span style="font-size:11px;float:none">Last updated: 2016-11-04 17:50 PDT</span></p>
<p>Apache Beam (incubating) provides a portable API layer for building sophisticated data-parallel processing engines that may be executed across a diversity of exeuction engines, or <i>runners</i>. The core concepts of this layer are based upon the Beam Model (formerly referred to as the <a href="http://www.vldb.org/pvldb/vol8/p1792-Akidau.pdf">Dataflow Model</a>), and implemented to varying degrees in each Beam runner. To help clarify the capabilities of individual runners, we\u2019ve created the capability matrix below.</p>
@@ -338,7 +338,7 @@
- <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"><center><b>~</b></center>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
</td>
</tr>
@@ -483,7 +483,7 @@
- <td width="25%" class="cap-summary" style="background-color:#8cf;border-color:#37d"><center><b>~</b></center>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
</td>
</tr>
@@ -508,7 +508,7 @@
- <td width="25%" class="cap-summary" style="background-color:#8cf;border-color:#37d"><center><b>~</b></center>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
</td>
</tr>
@@ -533,7 +533,7 @@
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#37d"><center><b>✕</b></center>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
</td>
</tr>
@@ -558,7 +558,7 @@
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#37d"><center><b>✕</b></center>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
</td>
</tr>
@@ -583,7 +583,7 @@
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#37d"><center><b>✕</b></center>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
</td>
</tr>
@@ -608,7 +608,7 @@
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#37d"><center><b>✕</b></center>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
</td>
</tr>
@@ -1007,7 +1007,7 @@
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><center><b>Partially: group by window in batch only</b></center><br />Uses Spark's groupByKey for grouping. Grouping by window is currently only supported in batch.
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><center><b>Partially: fully supported in batch mode</b></center><br />Using Spark's <tt>groupByKey</tt>. GroupByKey with multiple trigger firings in streaming mode is a work in progress.
</td>
</tr>
@@ -1057,7 +1057,7 @@
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><center><b>Yes: fully supported</b></center><br />Supports GroupedValues, Globally and PerKey.
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><center><b>Yes: fully supported</b></center><br />Using Spark's <tt>combineByKey</tt> and <tt>aggregate</tt> functions.
</td>
</tr>
@@ -1107,7 +1107,7 @@
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><center><b>Partially: not supported in streaming</b></center><br />Side input is actually a broadcast variable in Spark so it can't be updated during the life of a job. Spark-runner implementation of side input is more of an immutable, static, side input.
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><center><b>Yes: fully supported</b></center><br />Using Spark's broadcast variables. In streaming mode, side inputs may update but only between micro-batches.
</td>
</tr>
@@ -1157,7 +1157,7 @@
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><center><b>Partially: streaming requires more testing</b></center><br />Uses Spark's <tt>AccumulatorParam</tt> mechanism
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><center><b>Partially: may overcount when tasks are retried in transformations.</b></center><br />supported via <tt>AccumulatorParam</tt> mechanism. If a task retries, and the accumulator is not within a Spark "Action", an overcount is possible.
</td>
</tr>
@@ -1252,7 +1252,7 @@
- <td width="25%" class="cap" style="background-color:#8cf;border-color:#37d"><center><b>Partially: currently only supported in batch</b></center><br />
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><center><b>Yes: supported</b></center><br />
</td>
</tr>
@@ -1277,7 +1277,7 @@
- <td width="25%" class="cap" style="background-color:#8cf;border-color:#37d"><center><b>Partially: currently only supported in batch</b></center><br />
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><center><b>Yes: supported</b></center><br />
</td>
</tr>
@@ -1302,7 +1302,7 @@
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#37d"><center><b>No: pending Spark engine support</b></center><br />
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><center><b>Yes: supported</b></center><br />
</td>
</tr>
@@ -1327,7 +1327,7 @@
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#37d"><center><b>No: pending Spark engine support</b></center><br />
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><center><b>Yes: supported</b></center><br />
</td>
</tr>
@@ -1352,7 +1352,7 @@
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#37d"><center><b>No: pending Spark engine support</b></center><br />
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><center><b>Yes: supported</b></center><br />
</td>
</tr>
@@ -1377,7 +1377,7 @@
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#37d"><center><b>No: pending Spark engine support</b></center><br />
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><center><b>Yes: supported</b></center><br />
</td>
</tr>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/feed.xml
----------------------------------------------------------------------
diff --git a/content/feed.xml b/content/feed.xml
index a6a5821..4ab11f7 100644
--- a/content/feed.xml
+++ b/content/feed.xml
@@ -6,8 +6,8 @@
</description>
<link>http://beam.incubator.apache.org/</link>
<atom:link href="http://beam.incubator.apache.org/feed.xml" rel="self" type="application/rss+xml"/>
- <pubDate>Fri, 04 Nov 2016 16:45:19 -0700</pubDate>
- <lastBuildDate>Fri, 04 Nov 2016 16:45:19 -0700</lastBuildDate>
+ <pubDate>Fri, 04 Nov 2016 17:50:55 -0700</pubDate>
+ <lastBuildDate>Fri, 04 Nov 2016 17:50:55 -0700</lastBuildDate>
<generator>Jekyll v3.2.0</generator>
<item>
@@ -51,7 +51,7 @@ from the Mobile Gaming example series.</p>
<p><a href="https://github.com/apache/incubator-beam/blob/master/examples/java8/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java#L177">LeaderBoard</a>
is part of the <a href="https://github.com/apache/incubator-beam/tree/master/examples/java8/src/main/java/org/apache/beam/examples/complete/game">Beam mobile gaming examples</a>
-(and <a href="http://beam.incubator.apache.org/use/walkthroughs/">walkthroughs</a>)
+(and <a href="/get-started/mobile-gaming-example/">walkthroughs</a>)
which produces a continuous accounting of user and team scores. User scores are
calculated over the lifetime of the program, while team scores are calculated
within fixed windows with a default duration of one hour. The LeaderBoard
@@ -64,7 +64,7 @@ be controlled within a test.</p>
<h2 id="writing-deterministic-tests-to-emulate-nondeterminism">Writing Deterministic Tests to Emulate Nondeterminism</h2>
<p>The Beam testing infrastructure provides the
-<a href="http://beam.incubator.apache.org/learn/sdks/javadoc/0.2.0-incubating/">PAssert</a>
+<a href="/documentation/sdks/javadoc/0.2.0-incubating/">PAssert</a>
methods, which assert properties about the contents of a PCollection from within
a pipeline. We have expanded this infrastructure to include
<a href="https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/TestStream.java">TestStream</a>,
@@ -312,7 +312,7 @@ TestStream.</p>
has enabled the testing of Pipelines which produce speculative and late panes.
This permits tests for all styles of pipeline to be expressed directly within the
Java SDK. If you have questions or comments, we\u2019d love to hear them on the
-<a href="http://beam.incubator.apache.org/use/mailing-lists/">mailing lists</a>.</p>
+<a href="/use/mailing-lists/">mailing lists</a>.</p>
</description>
<pubDate>Thu, 20 Oct 2016 11:00:00 -0700</pubDate>
<link>http://beam.incubator.apache.org/blog/2016/10/20/test-stream.html</link>
@@ -345,7 +345,7 @@ Java SDK. If you have questions or comments, we\u2019d love to hear them on the
<p>We rounded out the tutorial with live demonstrations of Beam running on Apache Spark, Apache Flink, the local runner, and DataFlow runner. Then, we brought in the big brainpower and had a Q and A session.</p>
-<p>If you\u2019re attending a conference, we encourage you to look for a Beam session. If you want to use these materials to give your own Beam talk or tutorial, we\u2019re happy to help you. In addition to this tutorial, we have <a href="/learn/presentation-materials/">other presentation materials</a>. You can reach out to us on the <a href="http://beam.incubator.apache.org/use/mailing-lists/">user mailing list</a>.</p>
+<p>If you\u2019re attending a conference, we encourage you to look for a Beam session. If you want to use these materials to give your own Beam talk or tutorial, we\u2019re happy to help you. In addition to this tutorial, we have <a href="/learn/presentation-materials/">other presentation materials</a>. You can reach out to us on the <a href="/use/mailing-lists/">user mailing list</a>.</p>
</description>
<pubDate>Tue, 11 Oct 2016 10:00:00 -0700</pubDate>
@@ -398,7 +398,7 @@ Java SDK. If you have questions or comments, we\u2019d love to hear them on the
<li>Presented at multiple existing meetups and are starting to organize some of our own.</li>
</ul>
-<p>While it\u2019s nice to reflect back on all we\u2019ve done, we\u2019re working full <em>stream</em> ahead towards a stable release and graduation from incubator. And we\u2019d love your help \u2013 join the <a href="http://beam.incubator.apache.org/use/mailing-lists/">mailing lists</a>, check out the <a href="http://beam.incubator.apache.org/contribute/contribution-guide/">contribution guide</a>, and grab a <a href="https://issues.apache.org/jira/browse/BEAM-520?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20in%20(newbie%2C%20starter)">starter task</a> from Jira!</p>
+<p>While it\u2019s nice to reflect back on all we\u2019ve done, we\u2019re working full <em>stream</em> ahead towards a stable release and graduation from incubator. And we\u2019d love your help \u2013 join the <a href="/use/mailing-lists/">mailing lists</a>, check out the <a href="/contribute/contribution-guide/">contribution guide</a>, and grab a <a href="https://issues.apache.org/jira/browse/BEAM-520?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20in%20(newbie%2C%20starter)">starter task</a> from Jira!</p>
</description>
<pubDate>Wed, 03 Aug 2016 00:00:01 -0700</pubDate>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/get-started/index.html
----------------------------------------------------------------------
diff --git a/content/get-started/index.html b/content/get-started/index.html
index 394e6bb..f69c8f7 100644
--- a/content/get-started/index.html
+++ b/content/get-started/index.html
@@ -156,7 +156,7 @@
<ul>
<li><a href="/use/wordcount-example">WordCount</a>: Simple example pipelines that demonstrate basic Beam programming, including debugging and testing</li>
- <li><a href="/use/walkthroughs">Mobile Gaming</a>: A series of more advanced pipelines that demonstrate use cases in the mobile gaming domain</li>
+ <li><a href="/use/mobile-gaming-example">Mobile Gaming</a>: A series of more advanced pipelines that demonstrate use cases in the mobile gaming domain</li>
</ul>
<h4 id="support">Support</h4>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/get-started/mobile-gaming-example/index.html
----------------------------------------------------------------------
diff --git a/content/get-started/mobile-gaming-example/index.html b/content/get-started/mobile-gaming-example/index.html
index e07a48a..5f20db3 100644
--- a/content/get-started/mobile-gaming-example/index.html
+++ b/content/get-started/mobile-gaming-example/index.html
@@ -197,7 +197,7 @@
<p>Because some of our example pipelines use data files (like logs from the game server) as input, the event timestamp for each game might be embedded in the data\u2013that is, it\u2019s a field in each data record. Those pipelines need to parse the event timestamp from each data record after reading it from the input file.</p>
-<p>For pipelines that read unbounded game data from an unbounded source, the data source sets the intrinsic <a href="/learn/programming-guide/#pctimestamps">timestamp</a> for each PCollection element to the appropriate event time.</p>
+<p>For pipelines that read unbounded game data from an unbounded source, the data source sets the intrinsic <a href="/documentation/programming-guide/#pctimestamps">timestamp</a> for each PCollection element to the appropriate event time.</p>
<p>The Mobile Game example pipelines vary in complexity, from simple batch analysis to more complex pipelines that can perform real-time analysis and abuse detection. This section walks you through each example and demonstrates how to use Beam features like windowing and triggers to expand your pipeline\u2019s capabilites.</p>
@@ -238,7 +238,7 @@
<p>This example uses batch processing, and the diagram\u2019s Y axis represents processing time: the pipeline processes events lower on the Y-axis first, and events higher up the axis later. The diagram\u2019s X axis represents the event time for each game event, as denoted by that event\u2019s timestamp. Note that the individual events in the diagram are not processed by the pipeline in the same order as they occurred (according to their timestamps).</p>
-<p>After reading the score events from the input file, the pipeline groups all of those user/score pairs together and sums the score values into one total value per unique user. <code class="highlighter-rouge">UserScore</code> encapsulates the core logic for that step as the <a href="/learn/programming-guide/#transforms-composite">user-defined composite transform</a> <code class="highlighter-rouge">ExtractAndSumScore</code>:</p>
+<p>After reading the score events from the input file, the pipeline groups all of those user/score pairs together and sums the score values into one total value per unique user. <code class="highlighter-rouge">UserScore</code> encapsulates the core logic for that step as the <a href="/documentation/programming-guide/#transforms-composite">user-defined composite transform</a> <code class="highlighter-rouge">ExtractAndSumScore</code>:</p>
<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">ExtractAndSumScore</span>
<span class="kd">extends</span> <span class="n">PTransform</span><span class="o"><</span><span class="n">PCollection</span><span class="o"><</span><span class="n">GameActionInfo</span><span class="o">>,</span> <span class="n">PCollection</span><span class="o"><</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>>></span> <span class="o">{</span>
@@ -361,7 +361,7 @@
<p><strong>Note:</strong> As is shown in the diagram above, using windowing produces an <em>independent total for every interval</em> (in this case, each hour). <code class="highlighter-rouge">HourlyTeamScore</code> doesn\u2019t provide a running total for the entire data set at each hour\u2013it provides the total score for all the events that occurred <em>only within that hour</em>.</p>
</blockquote>
-<p>Beam\u2019s windowing feature uses the <a href="/learn/programming-guide/#pctimestamps">intrinsic timestamp information</a> attached to each element of a <code class="highlighter-rouge">PCollection</code>. Because we want our pipeline to window based on <em>event time</em>, we <strong>must first extract the timestamp</strong> that\u2019s embedded in each data record apply it to the corresponding element in the <code class="highlighter-rouge">PCollection</code> of score data. Then, the pipeline can <strong>apply the windowing function</strong> to divide the <code class="highlighter-rouge">PCollection</code> into logical windows.</p>
+<p>Beam\u2019s windowing feature uses the <a href="/documentation/programming-guide/#pctimestamps">intrinsic timestamp information</a> attached to each element of a <code class="highlighter-rouge">PCollection</code>. Because we want our pipeline to window based on <em>event time</em>, we <strong>must first extract the timestamp</strong> that\u2019s embedded in each data record apply it to the corresponding element in the <code class="highlighter-rouge">PCollection</code> of score data. Then, the pipeline can <strong>apply the windowing function</strong> to divide the <code class="highlighter-rouge">PCollection</code> into logical windows.</p>
<p>Here\u2019s the code, which shows how <code class="highlighter-rouge">HourlyTeamScore</code> uses the <a href="https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/WithTimestamps.java">WithTimestamps</a> and <a href="https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/windowing/Window.java">Window</a> transforms to perform these operations:</p>
@@ -486,7 +486,7 @@
<p>We want our pipeline to output a running total score for each user for every ten minutes that the pipeline runs. This calculation doesn\u2019t consider <em>when</em> the actual score was generated by the user\u2019s play instance; it simply outputs the sum of all the scores for that user that have arrived in the pipeline to date. Late data gets included in the calculation whenever it happens to arrive in the pipeline as it\u2019s running.</p>
-<p>Because we want all the data that has arrived in the pipeline every time we update our calculation, we have the pipeline consider all of the user score data in a <strong>single global window</strong>. The single global window is unbounded, but we can specify a kind of temporary cut-off point for each ten-minute calculation by using a processing time <a href="/learn/programming-guide/#triggers">trigger</a>.</p>
+<p>Because we want all the data that has arrived in the pipeline every time we update our calculation, we have the pipeline consider all of the user score data in a <strong>single global window</strong>. The single global window is unbounded, but we can specify a kind of temporary cut-off point for each ten-minute calculation by using a processing time <a href="/documentation/programming-guide/#triggers">trigger</a>.</p>
<p>When we specify a ten-minute processing time trigger for the single global window, the effect is that the pipeline effectively takes a \u201csnapshot\u201d of the contents of the window every time the trigger fires (which it does at ten-minute intervals). Since we\u2019re using a single global window, each snapshot contains all the data collected <em>to that point in time</em>. The following diagram shows the effects of using a processing time trigger on the single global window:</p>
@@ -536,7 +536,7 @@
<p>Because we consider each hour individually, we can apply fixed-time windowing to our input data, just like in <code class="highlighter-rouge">HourlyTeamScore</code>. To provide the speculative updates and updates on late data, we\u2019ll specify additional trigger parameters. The trigger will cause each window to calculate and emit results at an interval we specify (in this case, every five minutes), and also to keep triggering after the window is considered \u201ccomplete\u201d to account for late data. Just like the user score calculation, we\u2019ll set the trigger to accumulating mode to ensure that we get a running sum for each hour-long window.</p>
-<p>The triggers for speculative updates and late data help with the problem of <a href="/learn/programming-guide/#windowing">time skew</a>. Events in the pipeline aren\u2019t necessarily processed in the order in which they actually occurred according to their timestamps; they may arrive in the pipeline out of order, or late (in our case, because they were generated while the user\u2019s phone was out of contact with a network). Beam needs a way to determine when it can reasonably assume that it has \u201call\u201d of the data in a given window: this is called the <em>watermark</em>.</p>
+<p>The triggers for speculative updates and late data help with the problem of <a href="/documentation/programming-guide/#windowing">time skew</a>. Events in the pipeline aren\u2019t necessarily processed in the order in which they actually occurred according to their timestamps; they may arrive in the pipeline out of order, or late (in our case, because they were generated while the user\u2019s phone was out of contact with a network). Beam needs a way to determine when it can reasonably assume that it has \u201call\u201d of the data in a given window: this is called the <em>watermark</em>.</p>
<p>In an ideal world, all data would be processed immediately when it occurs, so the processing time would be equal to (or at least have a linear relationship to) the event time. However, because distributed systems contain some inherent inaccuracy (like our late-reporting phones), Beam often uses a heuristic watermark.</p>
@@ -613,7 +613,7 @@
<p>To determine whether or not a score is \u201cabnormally\u201d high, <code class="highlighter-rouge">GameStats</code> calculates the average of every score in that fixed-time window, and then checks each score individual score against the average score multiplied by an arbitrary weight factor (in our case, 2.5). Thus, any score more than 2.5 times the average is deemed to be the product of spam. The <code class="highlighter-rouge">GameStats</code> pipeline tracks a list of \u201cspam\u201d users and filters those users out of the team score calculations for the team leader board.</p>
-<p>Since the average depends on the pipeline data, we need to calculate it, and then use that calculated data in a subsequent <code class="highlighter-rouge">ParDo</code> transform that filters scores that exceed the weighted value. To do this, we can pass the calculated average to as a <a href="/learn/programming-guide/#transforms-sideio">side input</a> to the filtering <code class="highlighter-rouge">ParDo</code>.</p>
+<p>Since the average depends on the pipeline data, we need to calculate it, and then use that calculated data in a subsequent <code class="highlighter-rouge">ParDo</code> transform that filters scores that exceed the weighted value. To do this, we can pass the calculated average to as a <a href="/documentation/programming-guide/#transforms-sideio">side input</a> to the filtering <code class="highlighter-rouge">ParDo</code>.</p>
<p>The following code example shows the composite transform that handles abuse detection. The transform uses the <code class="highlighter-rouge">Sum.integersPerKey</code> transform to sum all scores per user, and then the <code class="highlighter-rouge">Mean.globally</code> transform to determine the average score for all users. Once that\u2019s been calculated (as a <code class="highlighter-rouge">PCollectionView</code> singleton), we can pass it to the filtering <code class="highlighter-rouge">ParDo</code> using <code class="highlighter-rouge">.withSideInputs</code>:</p>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/get-started/wordcount-example/index.html
----------------------------------------------------------------------
diff --git a/content/get-started/wordcount-example/index.html b/content/get-started/wordcount-example/index.html
index 5fe26fb..3331215 100644
--- a/content/get-started/wordcount-example/index.html
+++ b/content/get-started/wordcount-example/index.html
@@ -259,7 +259,7 @@ Figure 1: The pipeline data flow.</p>
</div>
</li>
<li>
- <p>A <a href="/learn/programming-guide/#transforms-pardo">ParDo</a> transform that invokes a <code class="highlighter-rouge">DoFn</code> (defined in-line as an anonymous class) on each element that tokenizes the text lines into individual words. The input for this transform is the <code class="highlighter-rouge">PCollection</code> of text lines generated by the previous <code class="highlighter-rouge">TextIO.Read</code> transform. The <code class="highlighter-rouge">ParDo</code> transform outputs a new <code class="highlighter-rouge">PCollection</code>, where each element represents an individual word in the text.</p>
+ <p>A <a href="/documentation/programming-guide/#transforms-pardo">ParDo</a> transform that invokes a <code class="highlighter-rouge">DoFn</code> (defined in-line as an anonymous class) on each element that tokenizes the text lines into individual words. The input for this transform is the <code class="highlighter-rouge">PCollection</code> of text lines generated by the previous <code class="highlighter-rouge">TextIO.Read</code> transform. The <code class="highlighter-rouge">ParDo</code> transform outputs a new <code class="highlighter-rouge">PCollection</code>, where each element represents an individual word in the text.</p>
<div class="language-java highlighter-rouge"><pre class="highlight"><code> <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"ExtractWords"</span><span class="o">,</span> <span class="n">ParDo</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="k">new</span> <span class="n">DoFn</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">>()</span> <span class="o">{</span>
<span class="nd">@ProcessElement</span>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/index.html
----------------------------------------------------------------------
diff --git a/content/index.html b/content/index.html
index 5bc913f..6d9a0b9 100644
--- a/content/index.html
+++ b/content/index.html
@@ -208,7 +208,7 @@ The Apache Beam project is in the process of bootstrapping. This includes the we
<p>Interested in working with Apache Beam? Great! Here\u2019s how to get started:</p>
<ul>
- <li>If you are interested in using Beam for your data processing tasks, use the <a href="/get-started/">Get Started</a> section for an overview, quickstart, and examples. Then dive into the <a href="/documentatiion/">Documentation</a> section to get up to speed on the details.</li>
+ <li>If you are interested in using Beam for your data processing tasks, use the <a href="/get-started/">Get Started</a> section for an overview, quickstart, and examples. Then dive into the <a href="/documentation/">Documentation</a> section to get up to speed on the details.</li>
<li>If you\u2019re interested in contributing to the Beam SDKs or Runners, start with the <a href="/contribute/">Contribute</a> section.</li>
</ul>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/learn/index.html
----------------------------------------------------------------------
diff --git a/content/learn/index.html b/content/learn/index.html
new file mode 100644
index 0000000..922447a
--- /dev/null
+++ b/content/learn/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/documentation/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/documentation/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/documentation/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/documentation/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/learn/programming-guide/index.html
----------------------------------------------------------------------
diff --git a/content/learn/programming-guide/index.html b/content/learn/programming-guide/index.html
new file mode 100644
index 0000000..14bdce8
--- /dev/null
+++ b/content/learn/programming-guide/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/documentation/programming-guide/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/documentation/programming-guide/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/documentation/programming-guide/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/documentation/programming-guide/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/learn/resources/index.html
----------------------------------------------------------------------
diff --git a/content/learn/resources/index.html b/content/learn/resources/index.html
new file mode 100644
index 0000000..fe3ad29
--- /dev/null
+++ b/content/learn/resources/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/documentation/resources/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/documentation/resources/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/documentation/resources/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/documentation/resources/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/learn/runners/capability-matrix/index.html
----------------------------------------------------------------------
diff --git a/content/learn/runners/capability-matrix/index.html b/content/learn/runners/capability-matrix/index.html
new file mode 100644
index 0000000..48f5824
--- /dev/null
+++ b/content/learn/runners/capability-matrix/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/documentation/runners/capability-matrix/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/documentation/runners/capability-matrix/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/documentation/runners/capability-matrix/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/documentation/runners/capability-matrix/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/learn/runners/dataflow/index.html
----------------------------------------------------------------------
diff --git a/content/learn/runners/dataflow/index.html b/content/learn/runners/dataflow/index.html
new file mode 100644
index 0000000..941d0a0
--- /dev/null
+++ b/content/learn/runners/dataflow/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/documentation/runners/dataflow/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/documentation/runners/dataflow/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/documentation/runners/dataflow/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/documentation/runners/dataflow/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/learn/runners/direct/index.html
----------------------------------------------------------------------
diff --git a/content/learn/runners/direct/index.html b/content/learn/runners/direct/index.html
new file mode 100644
index 0000000..563d208
--- /dev/null
+++ b/content/learn/runners/direct/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/documentation/runners/direct/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/documentation/runners/direct/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/documentation/runners/direct/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/documentation/runners/direct/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/learn/runners/flink/index.html
----------------------------------------------------------------------
diff --git a/content/learn/runners/flink/index.html b/content/learn/runners/flink/index.html
new file mode 100644
index 0000000..f793aa1
--- /dev/null
+++ b/content/learn/runners/flink/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/documentation/runners/flink/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/documentation/runners/flink/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/documentation/runners/flink/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/documentation/runners/flink/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/learn/runners/index.html
----------------------------------------------------------------------
diff --git a/content/learn/runners/index.html b/content/learn/runners/index.html
new file mode 100644
index 0000000..770b3f4
--- /dev/null
+++ b/content/learn/runners/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/documentation/runners/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/documentation/runners/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/documentation/runners/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/documentation/runners/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/learn/runners/spark/index.html
----------------------------------------------------------------------
diff --git a/content/learn/runners/spark/index.html b/content/learn/runners/spark/index.html
new file mode 100644
index 0000000..9a2ab4b
--- /dev/null
+++ b/content/learn/runners/spark/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/documentation/runners/spark/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/documentation/runners/spark/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/documentation/runners/spark/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/documentation/runners/spark/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/learn/sdks/index.html
----------------------------------------------------------------------
diff --git a/content/learn/sdks/index.html b/content/learn/sdks/index.html
new file mode 100644
index 0000000..62e8620
--- /dev/null
+++ b/content/learn/sdks/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/documentation/sdks/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/documentation/sdks/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/documentation/sdks/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/documentation/sdks/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/learn/sdks/java/index.html
----------------------------------------------------------------------
diff --git a/content/learn/sdks/java/index.html b/content/learn/sdks/java/index.html
new file mode 100644
index 0000000..60c42ca
--- /dev/null
+++ b/content/learn/sdks/java/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/documentation/sdks/java/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/documentation/sdks/java/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/documentation/sdks/java/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/documentation/sdks/java/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/learn/sdks/javadoc/index.html
----------------------------------------------------------------------
diff --git a/content/learn/sdks/javadoc/index.html b/content/learn/sdks/javadoc/index.html
new file mode 100644
index 0000000..f5aaa7b
--- /dev/null
+++ b/content/learn/sdks/javadoc/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/documentation/sdks/javadoc/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/documentation/sdks/javadoc/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/documentation/sdks/javadoc/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/documentation/sdks/javadoc/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/project/logos/index.html
----------------------------------------------------------------------
diff --git a/content/project/logos/index.html b/content/project/logos/index.html
new file mode 100644
index 0000000..3f395c6
--- /dev/null
+++ b/content/project/logos/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/contribute/logos/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/contribute/logos/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/contribute/logos/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/contribute/logos/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/project/team/index.html
----------------------------------------------------------------------
diff --git a/content/project/team/index.html b/content/project/team/index.html
new file mode 100644
index 0000000..d13d4b4
--- /dev/null
+++ b/content/project/team/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/contribute/team/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/contribute/team/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/contribute/team/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/contribute/team/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/use/beam-overview/index.html
----------------------------------------------------------------------
diff --git a/content/use/beam-overview/index.html b/content/use/beam-overview/index.html
new file mode 100644
index 0000000..a6cca54
--- /dev/null
+++ b/content/use/beam-overview/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/get-started/beam-overview/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/get-started/beam-overview/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/get-started/beam-overview/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/get-started/beam-overview/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/use/index.html
----------------------------------------------------------------------
diff --git a/content/use/index.html b/content/use/index.html
new file mode 100644
index 0000000..748b95e
--- /dev/null
+++ b/content/use/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/get-started/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/get-started/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/get-started/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/get-started/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/use/issue-tracking/index.html
----------------------------------------------------------------------
diff --git a/content/use/issue-tracking/index.html b/content/use/issue-tracking/index.html
new file mode 100644
index 0000000..5a14aaa
--- /dev/null
+++ b/content/use/issue-tracking/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/get-started/support/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/get-started/support/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/get-started/support/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/get-started/support/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/use/mailing-lists/index.html
----------------------------------------------------------------------
diff --git a/content/use/mailing-lists/index.html b/content/use/mailing-lists/index.html
new file mode 100644
index 0000000..5a14aaa
--- /dev/null
+++ b/content/use/mailing-lists/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/get-started/support/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/get-started/support/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/get-started/support/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/get-started/support/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/use/quickstart/index.html
----------------------------------------------------------------------
diff --git a/content/use/quickstart/index.html b/content/use/quickstart/index.html
new file mode 100644
index 0000000..c7b8fc9
--- /dev/null
+++ b/content/use/quickstart/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/get-started/quickstart/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/get-started/quickstart/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/get-started/quickstart/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/get-started/quickstart/"</script>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/d384fc51/content/use/releases/index.html
----------------------------------------------------------------------
diff --git a/content/use/releases/index.html b/content/use/releases/index.html
new file mode 100644
index 0000000..def15c3
--- /dev/null
+++ b/content/use/releases/index.html
@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting\u2026</title>
+<link rel="canonical" href="http://beam.incubator.apache.org/get-started/releases/">
+<meta http-equiv="refresh" content="0; url=http://beam.incubator.apache.org/get-started/releases/">
+<h1>Redirecting\u2026</h1>
+<a href="http://beam.incubator.apache.org/get-started/releases/">Click here if you are not redirected.</a>
+<script>location="http://beam.incubator.apache.org/get-started/releases/"</script>
+</html>
[4/4] incubator-beam-site git commit: This closes #66
Posted by da...@apache.org.
This closes #66
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/228a6c91
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/tree/228a6c91
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/diff/228a6c91
Branch: refs/heads/asf-site
Commit: 228a6c91d6e3111ed6d7e49de55972442a291baf
Parents: e96b07f d384fc5
Author: Davor Bonaci <da...@google.com>
Authored: Fri Nov 4 17:52:15 2016 -0700
Committer: Davor Bonaci <da...@google.com>
Committed: Fri Nov 4 17:52:15 2016 -0700
----------------------------------------------------------------------
Rakefile | 4 ++-
_config_test.yml | 5 +++
_includes/footer.html | 2 +-
_includes/head.html | 2 +-
_includes/header.html | 2 +-
_posts/2016-08-03-six-months.md | 2 +-
.../2016-10-12-strata-hadoop-world-and-beam.md | 2 +-
_posts/2016-10-20-test-stream.md | 6 ++--
blog/index.md | 2 +-
coming-soon.md | 2 +-
.../10/11/strata-hadoop-world-and-beam.html | 2 +-
content/blog/2016/08/03/six-months.html | 2 +-
content/blog/2016/10/20/test-stream.html | 6 ++--
content/capability-matrix/index.html | 10 ++++++
.../contribute/contribution-guide/index.html | 4 +--
.../runners/capability-matrix/index.html | 36 ++++++++++----------
content/feed.xml | 14 ++++----
content/get-started/index.html | 2 +-
.../mobile-gaming-example/index.html | 12 +++----
.../get-started/wordcount-example/index.html | 2 +-
content/index.html | 2 +-
content/learn/index.html | 10 ++++++
content/learn/programming-guide/index.html | 10 ++++++
content/learn/resources/index.html | 10 ++++++
.../learn/runners/capability-matrix/index.html | 10 ++++++
content/learn/runners/dataflow/index.html | 10 ++++++
content/learn/runners/direct/index.html | 10 ++++++
content/learn/runners/flink/index.html | 10 ++++++
content/learn/runners/index.html | 10 ++++++
content/learn/runners/spark/index.html | 10 ++++++
content/learn/sdks/index.html | 10 ++++++
content/learn/sdks/java/index.html | 10 ++++++
content/learn/sdks/javadoc/index.html | 10 ++++++
content/project/logos/index.html | 10 ++++++
content/project/team/index.html | 10 ++++++
content/use/beam-overview/index.html | 10 ++++++
content/use/index.html | 10 ++++++
content/use/issue-tracking/index.html | 10 ++++++
content/use/mailing-lists/index.html | 10 ++++++
content/use/quickstart/index.html | 10 ++++++
content/use/releases/index.html | 10 ++++++
contribute/contribution-guide.md | 8 ++---
contribute/index.md | 2 +-
contribute/logos.md | 9 ++---
contribute/release-guide.md | 14 ++++----
contribute/team.md | 5 +--
contribute/work-in-progress.md | 6 ++--
documentation/index.md | 7 ++--
documentation/pipelines/create-your-pipeline.md | 2 +-
documentation/pipelines/design-your-pipeline.md | 2 +-
documentation/pipelines/test-your-pipeline.md | 2 +-
documentation/programming-guide.md | 5 +--
documentation/resources.md | 4 +--
documentation/runners/capability-materix.md | 5 +--
documentation/runners/dataflow.md | 2 +-
documentation/runners/direct.md | 2 +-
documentation/runners/flink.md | 2 +-
documentation/runners/index.md | 2 +-
documentation/runners/spark.md | 2 +-
documentation/sdks/index.md | 2 +-
documentation/sdks/java.md | 6 ++--
documentation/sdks/javadoc/index.md | 2 +-
get-started/beam-overview.md | 7 ++--
get-started/downloads.md | 2 +-
get-started/index.md | 9 ++---
get-started/mobile-gaming-example.md | 24 ++++++-------
get-started/quickstart.md | 5 +--
get-started/releases.md | 5 +--
get-started/support.md | 3 ++
get-started/wordcount-example.md | 2 +-
index.md | 6 ++--
71 files changed, 352 insertions(+), 123 deletions(-)
----------------------------------------------------------------------