You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by ja...@apache.org on 2016/06/20 22:14:13 UTC
[4/5] incubator-beam-site git commit: Fixed HTML errors;
added link and HTML tests via rake
Fixed HTML errors; added link and HTML tests via rake
Adding gemfile.lock to gitignore
Removing Gemfile.lock
Remove Rakefile from static build
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/2a61d388
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/tree/2a61d388
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/diff/2a61d388
Branch: refs/heads/asf-site
Commit: 2a61d388f762b02cf40071a84211a55d20821685
Parents: 2c0a36d
Author: James Malone <ja...@gmail.com>
Authored: Mon Jun 20 13:40:21 2016 -0700
Committer: James Malone <ja...@gmail.com>
Committed: Mon Jun 20 14:54:05 2016 -0700
----------------------------------------------------------------------
.gitignore | 1 +
Gemfile | 3 +
Rakefile | 10 +
_config.yml | 2 +-
_includes/capability-matrix-row-blog.md | 2 +-
_includes/capability-matrix-row-full.md | 2 +-
_includes/capability-matrix-row-summary.md | 2 +-
_pages/blog.md | 2 +-
_pages/material.md | 4 +-
_pages/public-meetings.md | 2 +-
_posts/2016-02-22-beam-has-a-logo.markdown | 4 +-
coming-soon.md | 13 +-
.../2016/03/17/capability-matrix.html | 324 ++++++---
.../2016/04/03/presentation-materials.html | 2 +-
.../website/2016/02/22/beam-has-a-logo.html | 4 +-
.../blog/2016/05/18/splitAtFraction-method.html | 2 +-
.../05/27/where-is-my-pcollection-dot-map.html | 12 +-
.../06/13/flink-batch-runner-milestone.html | 8 +-
content/blog/index.html | 19 +-
content/capability-matrix/index.html | 650 ++++++++++++-------
content/coming-soon.html | 13 +-
content/contribution-guide/index.html | 45 +-
content/docs/index.html | 12 +-
content/feed.xml | 360 ++++++----
content/index.html | 13 +-
content/material/index.html | 36 +-
content/presentation-materials/index.html | 10 +-
content/public-meetings/index.html | 4 +-
content/releases/index.html | 7 +-
content/source_repository/index.html | 9 +-
docs/index.md | 12 +-
index.md | 13 +-
32 files changed, 1024 insertions(+), 578 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/.gitignore
----------------------------------------------------------------------
diff --git a/.gitignore b/.gitignore
index 45c1505..666de10 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,4 @@
_site
.sass-cache
.jekyll-metadata
+Gemfile.lock
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/Gemfile
----------------------------------------------------------------------
diff --git a/Gemfile b/Gemfile
new file mode 100644
index 0000000..bb67505
--- /dev/null
+++ b/Gemfile
@@ -0,0 +1,3 @@
+source 'https://rubygems.org'
+gem 'jekyll'
+gem 'html-proofer'
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/Rakefile
----------------------------------------------------------------------
diff --git a/Rakefile b/Rakefile
new file mode 100644
index 0000000..9778c98
--- /dev/null
+++ b/Rakefile
@@ -0,0 +1,10 @@
+require 'html-proofer'
+
+task :test do
+ sh "bundle exec jekyll build"
+ HTMLProofer.check_directory("./content", {
+ :allow_hash_href => true,
+ :check_html => true,
+ :file_ignore => [/javadoc/]
+ }).run
+end
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/_config.yml
----------------------------------------------------------------------
diff --git a/_config.yml b/_config.yml
index ed94c3c..697a7f8 100644
--- a/_config.yml
+++ b/_config.yml
@@ -38,7 +38,7 @@ collections:
- beam_team
# Things to ignore in the build
-exclude: ['README.md', 'Gemfile.lock', 'Gemfile']
+exclude: ['README.md', 'Gemfile.lock', 'Gemfile', 'Rakefile']
# Downloads directory
downloads: downloads
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/_includes/capability-matrix-row-blog.md
----------------------------------------------------------------------
diff --git a/_includes/capability-matrix-row-blog.md b/_includes/capability-matrix-row-blog.md
index bd3da68..5b43d9f 100644
--- a/_includes/capability-matrix-row-blog.md
+++ b/_includes/capability-matrix-row-blog.md
@@ -1 +1 @@
-<b><center>{% if val.l1 == 'Yes' %}✓{% elsif val.l1 == 'Partially' %}~{% else %}✕{% endif %}</center></b>
+<center><b>{% if val.l1 == 'Yes' %}✓{% elsif val.l1 == 'Partially' %}~{% else %}✕{% endif %}</b></center>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/_includes/capability-matrix-row-full.md
----------------------------------------------------------------------
diff --git a/_includes/capability-matrix-row-full.md b/_includes/capability-matrix-row-full.md
index 3734a98..9fa2d64 100644
--- a/_includes/capability-matrix-row-full.md
+++ b/_includes/capability-matrix-row-full.md
@@ -1 +1 @@
-<b><center>{{ val.l1 }}{% if val.l2 != '' %}: {{ val.l2 }}{% endif %}{% if val.jira %}<br>(<a href='https://issues.apache.org/jira/browse/{{ val.jira }}'>{{ val.jira }}</a>){% endif %}</center></b><br>{{ val.l3 }}
+<center><b>{{ val.l1 }}{% if val.l2 != '' %}: {{ val.l2 }}{% endif %}{% if val.jira %}<br>(<a href='https://issues.apache.org/jira/browse/{{ val.jira }}'>{{ val.jira }}</a>){% endif %}</b></center><br>{{ val.l3 }}
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/_includes/capability-matrix-row-summary.md
----------------------------------------------------------------------
diff --git a/_includes/capability-matrix-row-summary.md b/_includes/capability-matrix-row-summary.md
index 922802a..e1cc162 100644
--- a/_includes/capability-matrix-row-summary.md
+++ b/_includes/capability-matrix-row-summary.md
@@ -1 +1 @@
-<b><center>{% if val.l1 == 'Yes' %}✓{% elsif val.l1 == 'Partially' %}~{% else %}✕{% endif %}{% if val.jira %} (<a href='https://issues.apache.org/jira/browse/{{ val.jira }}'>{{ val.jira }}</a>){% endif %}</center></b>
+<center><b>{% if val.l1 == 'Yes' %}✓{% elsif val.l1 == 'Partially' %}~{% else %}✕{% endif %}{% if val.jira %} (<a href='https://issues.apache.org/jira/browse/{{ val.jira }}'>{{ val.jira }}</a>){% endif %}</b></center>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/_pages/blog.md
----------------------------------------------------------------------
diff --git a/_pages/blog.md b/_pages/blog.md
index 6a1f98b..36ada4a 100644
--- a/_pages/blog.md
+++ b/_pages/blog.md
@@ -24,7 +24,7 @@ for the project.
{% endcapture %}
{% if excerpt_words != content_words %}
<p>
-<a class="btn btn-default btn-sm" href="{{ post.url }}#read-more" role="button">
+<a class="btn btn-default btn-sm" href="{{ post.url }}" role="button">
Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"></span>
</a>
</p>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/_pages/material.md
----------------------------------------------------------------------
diff --git a/_pages/material.md b/_pages/material.md
index 23bc808..a372ce5 100644
--- a/_pages/material.md
+++ b/_pages/material.md
@@ -23,7 +23,7 @@ download it.
{% for type in site.data.logos.types %}
<div class="col-md-2">
<div class="row">
-<a href="{{ site.baseurl }}{{ site.data.logos.logo-location }}/{{ color[0] }}/{{ type }}/beam-logo-{{ color[0] }}-{{ type }}.svg" role="button"><img style="height: 60px" src="{{ site.baseurl }}{{ site.data.logos.logo-location }}/{{ color[0] }}/{{ type }}/beam-logo-{{ color[0] }}-{{ type }}.svg"></a>
+<a href="{{ site.baseurl }}{{ site.data.logos.logo-location }}/{{ color[0] }}/{{ type }}/beam-logo-{{ color[0] }}-{{ type }}.svg" role="button"><img style="height: 60px" src="{{ site.baseurl }}{{ site.data.logos.logo-location }}/{{ color[0] }}/{{ type }}/beam-logo-{{ color[0] }}-{{ type }}.svg" alt="beam-logo-{{ color[0] }}-{{ type }}.svg"></a>
</div><br>
</div>
{% endfor %}
@@ -43,7 +43,7 @@ available in a number of fixed sizes and are optimized for web use.
{% for type in site.data.logos.types %}
<div class="col-md-2">
<div class="row">
-<img style="height: 60px" src="{{ site.data.logos.logo-location }}/{{ color[0] }}/{{ type }}/beam-logo-{{ color[0] }}-{{ type }}.svg">
+<img style="height: 60px" src="{{ site.data.logos.logo-location }}/{{ color[0] }}/{{ type }}/beam-logo-{{ color[0] }}-{{ type }}.svg" alt="beam-logo-{{ color[0] }}-{{ type }}">
</div><br>
<div class="row">
{% for size in site.data.logos.sizes %}
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/_pages/public-meetings.md
----------------------------------------------------------------------
diff --git a/_pages/public-meetings.md b/_pages/public-meetings.md
index ead8c07..2b2a1dd 100644
--- a/_pages/public-meetings.md
+++ b/_pages/public-meetings.md
@@ -52,4 +52,4 @@ We are investigating methods, such as video conferencing and IRC chat, to allow
Public meetings include scheduled Apache Beam Dev/PPMC meetings, Meetup events, conference talks, and other events where the public meets to discuss Beam.
### How do I learn about new meetings?
-The Apache Beam community announces upcoming public meetings on the [dev@beam.incubator.apache.org](mailto:dev@beam.incubator.apache.org) mailing list. If you want to learn about new events, we recommend you [subscribe](dev-subscribe@beam.incubator.apache.org) to that list. If you are holding a public event, please send an email to the dev@ list.
+The Apache Beam community announces upcoming public meetings on the [dev@beam.incubator.apache.org](mailto:dev@beam.incubator.apache.org) mailing list. If you want to learn about new events, we recommend you [subscribe](mailto:dev-subscribe@beam.incubator.apache.org) to that list. If you are holding a public event, please send an email to the dev@ list.
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/_posts/2016-02-22-beam-has-a-logo.markdown
----------------------------------------------------------------------
diff --git a/_posts/2016-02-22-beam-has-a-logo.markdown b/_posts/2016-02-22-beam-has-a-logo.markdown
index f643a36..4ee569f 100644
--- a/_posts/2016-02-22-beam-has-a-logo.markdown
+++ b/_posts/2016-02-22-beam-has-a-logo.markdown
@@ -4,7 +4,7 @@ title: "Apache Beam has a logo!"
date: 2016-02-22 10:21:48 -0800
excerpt_separator: <!--more-->
categories: beam update website
-authors:
+authors:
- jamesmalone
---
@@ -18,7 +18,7 @@ now has a logo.
*drum roll* - **Presenting, the Apache Beam Logo!**
-<img src="{{ "/images/beam_logo_s.png" | prepend: site.baseurl }}">
+<img src="{{ "/images/beam_logo_s.png" | prepend: site.baseurl }}" alt="Apache Beam Logo">
We are excited about this logo because it is **simple**, **bright**, and shows the
unification of bath and streaming, as beams of light, within the 'B'. We will base
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/coming-soon.md
----------------------------------------------------------------------
diff --git a/coming-soon.md b/coming-soon.md
index 370af20..9c81b0b 100644
--- a/coming-soon.md
+++ b/coming-soon.md
@@ -1,17 +1,14 @@
---
layout: default
---
-<p>
- <div class="alert alert-info alert-dismissible" role="alert">
- <span class="glyphicon glyphicon-flag" aria-hidden="true"></span>
- <button type="button" class="close" data-dismiss="alert" aria-label="Close"><span aria-hidden="true">×</span></button>
- The Apache Beam project is in the process of bootstrapping. This includes the creation of project resources, the refactoring of the initial code submission, and the formulation of project documentation, planning, and design documents. For more information about Beam see the <a href="/getting_started/">getting started page</a>.
- </div>
-</p>
+<div class="alert alert-info alert-dismissible" role="alert">
+<span class="glyphicon glyphicon-flag" aria-hidden="true"></span>
+<button type="button" class="close" data-dismiss="alert" aria-label="Close"><span aria-hidden="true">×</span></button>
+The Apache Beam project is in the process of bootstrapping. This includes the creation of project resources, the refactoring of the initial code submission, and the formulation of project documentation, planning, and design documents. For more information about Beam see the <a href="/getting_started/">getting started page</a>.
+</div>
# Documentation Coming Soon
You've reached a page that's still in draft, or otherwise being developed! Please bear with us as we improve the documentation for Apache Beam.
[Go Back](/) to the main Beam site.
-
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/content/beam/capability/2016/03/17/capability-matrix.html
----------------------------------------------------------------------
diff --git a/content/beam/capability/2016/03/17/capability-matrix.html b/content/beam/capability/2016/03/17/capability-matrix.html
index 4d6ca5d..94f932f 100644
--- a/content/beam/capability/2016/03/17/capability-matrix.html
+++ b/content/beam/capability/2016/03/17/capability-matrix.html
@@ -154,19 +154,23 @@
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
</tr>
@@ -175,19 +179,23 @@
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"><center><b>~</b></center>
+</td>
</tr>
@@ -196,19 +204,23 @@
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
</tr>
@@ -217,19 +229,23 @@
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
</tr>
@@ -238,19 +254,23 @@
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"><center><b>~</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"><center><b>~</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"><center><b>~</b></center>
+</td>
</tr>
@@ -259,19 +279,23 @@
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"><center><b>~</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"><center><b>~</b></center>
+</td>
</tr>
@@ -280,19 +304,23 @@
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"><center><b>~</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ec3;border-color:#ca1"><center><b>✓</b></center>
+</td>
</tr>
@@ -301,19 +329,23 @@
- <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"><center><b>~</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"><center><b>~</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"><center><b>~</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#fe5;border-color:#ca1"><center><b>~</b></center>
+</td>
</tr>
@@ -322,19 +354,23 @@
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#ca1"><center><b>✕</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#ca1"><center><b>✕</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#ca1"><center><b>✕</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#ca1"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#ca1"><center><b>✕</b></center>
+</td>
</tr>
@@ -363,19 +399,23 @@
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
</tr>
@@ -384,19 +424,23 @@
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#8cf;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8cf;border-color:#37d"><center><b>~</b></center>
+</td>
</tr>
@@ -405,19 +449,23 @@
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#37d"><center><b>✕</b></center>
+</td>
</tr>
@@ -426,19 +474,23 @@
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#37d"><center><b>✕</b></center>
+</td>
</tr>
@@ -447,19 +499,23 @@
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#37d"><center><b>✕</b></center>
+</td>
</tr>
@@ -468,19 +524,23 @@
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#37d"><center><b>✕</b></center>
+</td>
</tr>
@@ -489,19 +549,23 @@
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#59f;border-color:#37d"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#37d"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#37d"><center><b>✕</b></center>
+</td>
</tr>
@@ -530,19 +594,23 @@
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"><center><b>✕</b></center>
+</td>
</tr>
@@ -551,19 +619,23 @@
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"><center><b>✕</b></center>
+</td>
</tr>
@@ -572,19 +644,23 @@
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
</tr>
@@ -593,19 +669,23 @@
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"><center><b>✕</b></center>
+</td>
</tr>
@@ -614,19 +694,23 @@
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"><center><b>✕</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"><center><b>✕</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"><center><b>✕</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"><center><b>✕</b></center>
+</td>
</tr>
@@ -635,19 +719,23 @@
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"><center><b>✕</b></center>
+</td>
</tr>
@@ -656,19 +744,23 @@
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#8c6;border-color:#6a4"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"><center><b>✕</b></center>
+</td>
</tr>
@@ -677,19 +769,23 @@
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"><center><b>✕</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"><center><b>✕</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"><center><b>✕</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#6a4"><center><b>✕</b></center>
+</td>
</tr>
@@ -718,19 +814,23 @@
- <td width="25%" class="cap-summary" style="background-color:#d77;border-color:#b55"></td>
+ <td width="25%" class="cap-summary" style="background-color:#d77;border-color:#b55"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#d77;border-color:#b55"></td>
+ <td width="25%" class="cap-summary" style="background-color:#d77;border-color:#b55"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#d77;border-color:#b55"></td>
+ <td width="25%" class="cap-summary" style="background-color:#d77;border-color:#b55"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#d77;border-color:#b55"></td>
+ <td width="25%" class="cap-summary" style="background-color:#d77;border-color:#b55"><center><b>✓</b></center>
+</td>
</tr>
@@ -739,19 +839,23 @@
- <td width="25%" class="cap-summary" style="background-color:#d77;border-color:#b55"></td>
+ <td width="25%" class="cap-summary" style="background-color:#d77;border-color:#b55"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#d77;border-color:#b55"></td>
+ <td width="25%" class="cap-summary" style="background-color:#d77;border-color:#b55"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#d77;border-color:#b55"></td>
+ <td width="25%" class="cap-summary" style="background-color:#d77;border-color:#b55"><center><b>✓</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#b55"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#b55"><center><b>✕</b></center>
+</td>
</tr>
@@ -760,19 +864,23 @@
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#b55"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#b55"><center><b>✕</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#b55"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#b55"><center><b>✕</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#b55"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#b55"><center><b>✕</b></center>
+</td>
- <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#b55"></td>
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#b55"><center><b>✕</b></center>
+</td>
</tr>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/content/beam/capability/2016/04/03/presentation-materials.html
----------------------------------------------------------------------
diff --git a/content/beam/capability/2016/04/03/presentation-materials.html b/content/beam/capability/2016/04/03/presentation-materials.html
index 5ee690d..469edcf 100644
--- a/content/beam/capability/2016/04/03/presentation-materials.html
+++ b/content/beam/capability/2016/04/03/presentation-materials.html
@@ -128,7 +128,7 @@
</li>
</ul>
-<p>As Apache Beam grows, so will this repository of presentation materials. We are excited to add new materials as the Apache Beam ecosystem grows with new runners, SDKs, and so on. If you are interested in contributing content or have a request, please see the <a href="/presentation-materials/">Apache Beam presentation materials</a> page or email the <a href="mailto:user@beam.incubator.apache.org"><code>user@beam.incubator.apache.org</code></a> mailing list with your ideas or questions.</p>
+<p>As Apache Beam grows, so will this repository of presentation materials. We are excited to add new materials as the Apache Beam ecosystem grows with new runners, SDKs, and so on. If you are interested in contributing content or have a request, please see the <a href="/presentation-materials/">Apache Beam presentation materials</a> page or email the <a href="mailto:user@beam.incubator.apache.org"><code class="highlighter-rouge">user@beam.incubator.apache.org</code></a> mailing list with your ideas or questions.</p>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/content/beam/update/website/2016/02/22/beam-has-a-logo.html
----------------------------------------------------------------------
diff --git a/content/beam/update/website/2016/02/22/beam-has-a-logo.html b/content/beam/update/website/2016/02/22/beam-has-a-logo.html
index 605cfda..05cd476 100644
--- a/content/beam/update/website/2016/02/22/beam-has-a-logo.html
+++ b/content/beam/update/website/2016/02/22/beam-has-a-logo.html
@@ -118,13 +118,13 @@ now has a logo.</p>
<p><em>drum roll</em> - <strong>Presenting, the Apache Beam Logo!</strong></p>
-<p><img src="/images/beam_logo_s.png" /></p>
+<p><img src="/images/beam_logo_s.png" alt="Apache Beam Logo" /></p>
<p>We are excited about this logo because it is <strong>simple</strong>, <strong>bright</strong>, and shows the
unification of bath and streaming, as beams of light, within the \u2018B\u2019. We will base
our future website and documentation design around this logo and its coloring. We
will also make various permutations and resolutions of this logo available in the
-coming weeks. For any questions or comments, send an email to the <code>dev@</code> email list
+coming weeks. For any questions or comments, send an email to the <code class="highlighter-rouge">dev@</code> email list
for Apache Beam.</p>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/content/blog/2016/05/18/splitAtFraction-method.html
----------------------------------------------------------------------
diff --git a/content/blog/2016/05/18/splitAtFraction-method.html b/content/blog/2016/05/18/splitAtFraction-method.html
index d79956c..9ddc339 100644
--- a/content/blog/2016/05/18/splitAtFraction-method.html
+++ b/content/blog/2016/05/18/splitAtFraction-method.html
@@ -114,7 +114,7 @@
<p>In a large batch processing job with many tasks executing in parallel, some of the tasks \u2013 the stragglers \u2013 can take a much longer time to complete than others, perhaps due to imperfect splitting of the work into parallel chunks when issuing the job. Typically, waiting for stragglers means that the overall job completes later than it should, and may also reserve too many machines that may be underutilized at the end. Cloud Dataflow\u2019s dynamic work rebalancing can mitigate stragglers in most cases.</p>
-<p>What I\u2019d like to highlight for the Apache Beam (incubating) community is that Cloud Dataflow\u2019s dynamic work rebalancing is implemented using <em>runner-specific</em> control logic on top of Beam\u2019s <em>runner-independent</em> <a href="https://github.com/apache/incubator-beam/blob/9fa97fb2491bc784df53fb0f044409dbbc2af3d7/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BoundedSource.java"><code>BoundedSource API</code></a>. Specifically, to steal work from a straggler, a runner need only call the reader\u2019s <a href="https://github.com/apache/incubator-beam/blob/3edae9b8b4d7afefb5c803c19bb0a1c21ebba89d/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BoundedSource.java#L266"><code>splitAtFraction method</code></a>. This will generate a new source containing leftover work, and then the runner can pass that source off to another idle worker. As Beam matures, I hope that other runners are interested in figuring out whether these APIs can help them improve performance, imple
menting dynamic work rebalancing, and collaborating on API changes that will help solve other pain points.</p>
+<p>What I\u2019d like to highlight for the Apache Beam (incubating) community is that Cloud Dataflow\u2019s dynamic work rebalancing is implemented using <em>runner-specific</em> control logic on top of Beam\u2019s <em>runner-independent</em> <a href="https://github.com/apache/incubator-beam/blob/9fa97fb2491bc784df53fb0f044409dbbc2af3d7/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BoundedSource.java"><code class="highlighter-rouge">BoundedSource API</code></a>. Specifically, to steal work from a straggler, a runner need only call the reader\u2019s <a href="https://github.com/apache/incubator-beam/blob/3edae9b8b4d7afefb5c803c19bb0a1c21ebba89d/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BoundedSource.java#L266"><code class="highlighter-rouge">splitAtFraction method</code></a>. This will generate a new source containing leftover work, and then the runner can pass that source off to another idle worker. As Beam matures, I hope that other runners are interested in figuring out whether
these APIs can help them improve performance, implementing dynamic work rebalancing, and collaborating on API changes that will help solve other pain points.</p>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/content/blog/2016/05/27/where-is-my-pcollection-dot-map.html
----------------------------------------------------------------------
diff --git a/content/blog/2016/05/27/where-is-my-pcollection-dot-map.html b/content/blog/2016/05/27/where-is-my-pcollection-dot-map.html
index 606dec1..a3157b7 100644
--- a/content/blog/2016/05/27/where-is-my-pcollection-dot-map.html
+++ b/content/blog/2016/05/27/where-is-my-pcollection-dot-map.html
@@ -6,7 +6,7 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Where's my PCollection.map()?</title>
+ <title>Where's my PCollection.map()?</title>
<meta name="description" content="Have you ever wondered why Beam has PTransforms for everything instead of having methods on PCollection? Take a look at the history that led to this (and oth...">
<link rel="stylesheet" href="/styles/site.css">
@@ -114,7 +114,7 @@
<p>Though Beam is relatively new, its design draws heavily on many years of experience with real-world pipelines. One of the primary inspirations is <a href="http://research.google.com/pubs/pub35650.html">FlumeJava</a>, which is Google\u2019s internal successor to MapReduce first introduced in 2009.</p>
-<p>The original FlumeJava API has methods like <code>count</code> and <code>parallelDo</code> on the PCollections. Though slightly more succinct, this approach has many disadvantages to extensibility. Every new user to FlumeJava wanted to add transforms, and adding them as methods to PCollection simply doesn\u2019t scale well. In contrast, a PCollection in Beam has a single <code>apply</code> method which takes any PTransform as an argument.</p>
+<p>The original FlumeJava API has methods like <code class="highlighter-rouge">count</code> and <code class="highlighter-rouge">parallelDo</code> on the PCollections. Though slightly more succinct, this approach has many disadvantages to extensibility. Every new user to FlumeJava wanted to add transforms, and adding them as methods to PCollection simply doesn\u2019t scale well. In contrast, a PCollection in Beam has a single <code class="highlighter-rouge">apply</code> method which takes any PTransform as an argument.</p>
<table class="table">
<tr>
@@ -138,7 +138,7 @@ PCollection<O> output = input.apply(Count.perElement())
<p>This is a more scalable approach for several reasons.</p>
<h2 id="where-to-draw-the-line">Where to draw the line?</h2>
-<p>Adding methods to PCollection forces a line to be drawn between operations that are \u201cuseful\u201d enough to merit this special treatment and those that are not. It is easy to make the case for flat map, group by key, and combine per key. But what about filter? Count? Approximate count? Approximate quantiles? Most frequent? WriteToMyFavoriteSource? Going too far down this path leads to a single enormous class that contains nearly everything one could want to do. (FlumeJava\u2019s PCollection class is over 5000 lines long with around 70 distinct operations, and it could have been <em>much</em> larger had we accepted every proposal.) Furthermore, since Java doesn\u2019t allow adding methods to a class, there is a sharp syntactic divide between those operations that are added to PCollection and those that aren\u2019t. A traditional way to share code is with a library of functions, but functions (in traditional languages like Java at least) are written prefix-style, which doesn\u2019t mix well wit
h the fluent builder style (e.g. <code>input.operation1().operation2().operation3()</code> vs. <code>operation3(operation1(input).operation2())</code>).</p>
+<p>Adding methods to PCollection forces a line to be drawn between operations that are \u201cuseful\u201d enough to merit this special treatment and those that are not. It is easy to make the case for flat map, group by key, and combine per key. But what about filter? Count? Approximate count? Approximate quantiles? Most frequent? WriteToMyFavoriteSource? Going too far down this path leads to a single enormous class that contains nearly everything one could want to do. (FlumeJava\u2019s PCollection class is over 5000 lines long with around 70 distinct operations, and it could have been <em>much</em> larger had we accepted every proposal.) Furthermore, since Java doesn\u2019t allow adding methods to a class, there is a sharp syntactic divide between those operations that are added to PCollection and those that aren\u2019t. A traditional way to share code is with a library of functions, but functions (in traditional languages like Java at least) are written prefix-style, which doesn\u2019t mix well wit
h the fluent builder style (e.g. <code class="highlighter-rouge">input.operation1().operation2().operation3()</code> vs. <code class="highlighter-rouge">operation3(operation1(input).operation2())</code>).</p>
<p>Instead in Beam we\u2019ve chosen a style that places all transforms\u2013whether they be primitive operations, composite operations bundled in the SDK, or part of an external library\u2013on equal footing. This also facilitates alternative implementations (which may even take different options) that are easily interchangeable.</p>
@@ -166,14 +166,14 @@ PCollection<O> output = input
</table>
<h2 id="configurability">Configurability</h2>
-<p>It makes for a fluent style to let values (PCollections) be the objects passed around and manipulated (i.e. the handles to the deferred execution graph), but it is the operations themselves that need to be composable, configurable, and extendable. Using PCollection methods for the operations doesn\u2019t scale well here, especially in a language without default or keyword arguments. For example, a ParDo operation can have any number of side inputs and side outputs, or a write operation may have configurations dealing with encoding and compression. One option is to separate these out into multiple overloads or even methods, but that exacerbates the problems above. (FlumeJava evolved over a dozen overloads of the <code>parallelDo</code> method!) Another option is to pass each method a configuration object that can be built up using more fluent idioms like the builder pattern, but at that point one might as well make the configuration object the operation itself, which is what Beam doe
s.</p>
+<p>It makes for a fluent style to let values (PCollections) be the objects passed around and manipulated (i.e. the handles to the deferred execution graph), but it is the operations themselves that need to be composable, configurable, and extendable. Using PCollection methods for the operations doesn\u2019t scale well here, especially in a language without default or keyword arguments. For example, a ParDo operation can have any number of side inputs and side outputs, or a write operation may have configurations dealing with encoding and compression. One option is to separate these out into multiple overloads or even methods, but that exacerbates the problems above. (FlumeJava evolved over a dozen overloads of the <code class="highlighter-rouge">parallelDo</code> method!) Another option is to pass each method a configuration object that can be built up using more fluent idioms like the builder pattern, but at that point one might as well make the configuration object the operation itse
lf, which is what Beam does.</p>
<h2 id="type-safety">Type Safety</h2>
-<p>Many operations can only be applied to collections whose elements are of a specific type. For example, the GroupByKey operation should only be applied to <code>PCollection<KV<K, V>></code>s. In Java at least, it\u2019s not possible to restrict methods based on the element type parameter alone. In FlumeJava, this led us to add a <code>PTable<K, V></code> subclassing <code>PCollection<KV<K, V>></code> to contain all the operations specific to PCollections of key-value pairs. This leads to the same question of which element types are special enough to merit being captured by PCollection subclasses. It is not very extensible for third parties and often requires manual downcasts/conversions (which can\u2019t be safely chained in Java) and special operations that produce these PCollection specializations.</p>
+<p>Many operations can only be applied to collections whose elements are of a specific type. For example, the GroupByKey operation should only be applied to <code class="highlighter-rouge">PCollection<KV<K, V>></code>s. In Java at least, it\u2019s not possible to restrict methods based on the element type parameter alone. In FlumeJava, this led us to add a <code class="highlighter-rouge">PTable<K, V></code> subclassing <code class="highlighter-rouge">PCollection<KV<K, V>></code> to contain all the operations specific to PCollections of key-value pairs. This leads to the same question of which element types are special enough to merit being captured by PCollection subclasses. It is not very extensible for third parties and often requires manual downcasts/conversions (which can\u2019t be safely chained in Java) and special operations that produce these PCollection specializations.</p>
<p>This is particularly inconvenient for transforms that produce outputs whose element types are the same as (or related to) their input\u2019s element types, requiring extra support to generate the right subclasses (e.g. a filter on a PTable should produce another PTable rather than just a raw PCollection of key-value pairs).</p>
-<p>Using PTransforms allows us to sidestep this entire issue. We can place arbitrary constraints on the context in which a transform may be used based on the type of its inputs; for instance GroupByKey is statically typed to only apply to a <code>PCollection<KV<K, V>></code>. The way this happens is generalizable to arbitrary shapes, without needing to introduce specialized types like PTable.</p>
+<p>Using PTransforms allows us to sidestep this entire issue. We can place arbitrary constraints on the context in which a transform may be used based on the type of its inputs; for instance GroupByKey is statically typed to only apply to a <code class="highlighter-rouge">PCollection<KV<K, V>></code>. The way this happens is generalizable to arbitrary shapes, without needing to introduce specialized types like PTable.</p>
<h2 id="reusability-and-structure">Reusability and Structure</h2>
<p>Though PTransforms are generally constructed at the site at which they\u2019re used, by pulling them out as separate objects one is able to store them and pass them around.</p>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/content/blog/2016/06/13/flink-batch-runner-milestone.html
----------------------------------------------------------------------
diff --git a/content/blog/2016/06/13/flink-batch-runner-milestone.html b/content/blog/2016/06/13/flink-batch-runner-milestone.html
index c23f566..c9780f1 100644
--- a/content/blog/2016/06/13/flink-batch-runner-milestone.html
+++ b/content/blog/2016/06/13/flink-batch-runner-milestone.html
@@ -112,20 +112,20 @@
<!--more-->
-<p>Before we start, though, let\u2019s quickly talk about the execution of Beam programs and how this is relevant to today\u2019s post. A Beam pipeline can contain bounded and unbounded sources. If the pipeline only contains bounded sources it can be executed in a batch fashion, if it contains some unbounded sources it must be executed in a streaming fashion. When executing a Beam pipeline on Flink, you don\u2019t have to choose the execution mode. Internally, the Flink runner either translates the pipeline to a Flink <code>DataSet</code> program or a <code>DataStream</code> program, depending on whether unbounded sources are used in the pipeline. In the following, when we say \u201cBatch runner\u201d what we are really talking about is the Flink runner being in batch execution mode.</p>
+<p>Before we start, though, let\u2019s quickly talk about the execution of Beam programs and how this is relevant to today\u2019s post. A Beam pipeline can contain bounded and unbounded sources. If the pipeline only contains bounded sources it can be executed in a batch fashion, if it contains some unbounded sources it must be executed in a streaming fashion. When executing a Beam pipeline on Flink, you don\u2019t have to choose the execution mode. Internally, the Flink runner either translates the pipeline to a Flink <code class="highlighter-rouge">DataSet</code> program or a <code class="highlighter-rouge">DataStream</code> program, depending on whether unbounded sources are used in the pipeline. In the following, when we say \u201cBatch runner\u201d what we are really talking about is the Flink runner being in batch execution mode.</p>
<h2 id="what-does-this-mean-for-users">What does this mean for users?</h2>
<p>Support for windowing was the last missing puzzle piece for making the Flink Batch runner compatible with the Beam model. With the latest change to the Batch runner users can now run any pipeline that only contains bounded sources and be certain that the results match those of the original reference-implementation runners that were provided by Google as part of the initial code drop coming from the Google Dataflow SDK.</p>
-<p>The most obvious part of the change is that windows can now be assigned to elements and that the runner respects these windows for the <code>GroupByKey</code> and <code>Combine</code> operations. A not-so-obvious change concerns side-inputs. In the Beam model, side inputs respect windows; when a value of the main input is being processed only the side input that corresponds to the correct window is available to the processing function, the <code>DoFn</code>.</p>
+<p>The most obvious part of the change is that windows can now be assigned to elements and that the runner respects these windows for the <code class="highlighter-rouge">GroupByKey</code> and <code class="highlighter-rouge">Combine</code> operations. A not-so-obvious change concerns side-inputs. In the Beam model, side inputs respect windows; when a value of the main input is being processed only the side input that corresponds to the correct window is available to the processing function, the <code class="highlighter-rouge">DoFn</code>.</p>
<p>Getting side-input semantics right is an important milestone in it\u2019s own because it allows to use a big suite of unit tests for verifying the correctness of a runner implementation. These tests exercise every obscure detail of the Beam programming model and verify that the results produced by a runner match what you would expect from a correct implementation. In the suite, side inputs are used to compare the expected result to the actual result. With these tests being executed regularly we can now be more confident that the implementation produces correct results for user-specified pipelines.</p>
<h2 id="under-the-hood">Under the Hood</h2>
-<p>The basis for the changes is the introduction of <code>WindowedValue</code> in the generated Flink transformations. Before, a Beam <code>PCollection<T></code> would be transformed to a <code>DataSet<T></code>. Now, we instead create a <code>DataSet<WindowedValue<T>></code>. The <code>WindowedValue<T></code> stores meta data about the value, such as the timestamp and the windows to which it was assigned.</p>
+<p>The basis for the changes is the introduction of <code class="highlighter-rouge">WindowedValue</code> in the generated Flink transformations. Before, a Beam <code class="highlighter-rouge">PCollection<T></code> would be transformed to a <code class="highlighter-rouge">DataSet<T></code>. Now, we instead create a <code class="highlighter-rouge">DataSet<WindowedValue<T>></code>. The <code class="highlighter-rouge">WindowedValue<T></code> stores meta data about the value, such as the timestamp and the windows to which it was assigned.</p>
-<p>With this basic change out of the way we just had to make sure that windows were respected for side inputs and that <code>Combine</code> and <code>GroupByKey</code> correctly handled windows. The tricky part there is the handling of merging windows such as session windows. For these we essentially emulate the behavior of a merging <code>WindowFn</code> in our own code.</p>
+<p>With this basic change out of the way we just had to make sure that windows were respected for side inputs and that <code class="highlighter-rouge">Combine</code> and <code class="highlighter-rouge">GroupByKey</code> correctly handled windows. The tricky part there is the handling of merging windows such as session windows. For these we essentially emulate the behavior of a merging <code class="highlighter-rouge">WindowFn</code> in our own code.</p>
<p>After we got side inputs working we could enable the aforementioned suite of tests to check how well the runner behaves with respect to the Beam model. As can be expected there were quite some discrepancies but we managed to resolve them all. In the process, we also slimmed down the runner implementation. For example, we removed all custom translations for sources and sinks and are now relying only on Beam code for these, thereby greatly reducing the maintenance overhead.</p>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/2a61d388/content/blog/index.html
----------------------------------------------------------------------
diff --git a/content/blog/index.html b/content/blog/index.html
index eee879d..f3e83c5 100644
--- a/content/blog/index.html
+++ b/content/blog/index.html
@@ -120,7 +120,7 @@ this year.</p>
<!-- Render a "read more" button if the post is longer than the excerpt -->
<p>
-<a class="btn btn-default btn-sm" href="/beam/release/2016/06/15/first-release.html#read-more" role="button">
+<a class="btn btn-default btn-sm" href="/beam/release/2016/06/15/first-release.html" role="button">
Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"></span>
</a>
</p>
@@ -136,7 +136,7 @@ Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"><
<!-- Render a "read more" button if the post is longer than the excerpt -->
<p>
-<a class="btn btn-default btn-sm" href="/blog/2016/06/13/flink-batch-runner-milestone.html#read-more" role="button">
+<a class="btn btn-default btn-sm" href="/blog/2016/06/13/flink-batch-runner-milestone.html" role="button">
Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"></span>
</a>
</p>
@@ -152,7 +152,7 @@ Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"><
<!-- Render a "read more" button if the post is longer than the excerpt -->
<p>
-<a class="btn btn-default btn-sm" href="/blog/2016/05/27/where-is-my-pcollection-dot-map.html#read-more" role="button">
+<a class="btn btn-default btn-sm" href="/blog/2016/05/27/where-is-my-pcollection-dot-map.html" role="button">
Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"></span>
</a>
</p>
@@ -168,7 +168,7 @@ Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"><
<!-- Render a "read more" button if the post is longer than the excerpt -->
<p>
-<a class="btn btn-default btn-sm" href="/blog/2016/05/18/splitAtFraction-method.html#read-more" role="button">
+<a class="btn btn-default btn-sm" href="/blog/2016/05/18/splitAtFraction-method.html" role="button">
Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"></span>
</a>
</p>
@@ -184,7 +184,7 @@ Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"><
<!-- Render a "read more" button if the post is longer than the excerpt -->
<p>
-<a class="btn btn-default btn-sm" href="/beam/capability/2016/04/03/presentation-materials.html#read-more" role="button">
+<a class="btn btn-default btn-sm" href="/beam/capability/2016/04/03/presentation-materials.html" role="button">
Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"></span>
</a>
</p>
@@ -200,7 +200,7 @@ Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"><
<!-- Render a "read more" button if the post is longer than the excerpt -->
<p>
-<a class="btn btn-default btn-sm" href="/beam/capability/2016/03/17/capability-matrix.html#read-more" role="button">
+<a class="btn btn-default btn-sm" href="/beam/capability/2016/03/17/capability-matrix.html" role="button">
Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"></span>
</a>
</p>
@@ -217,7 +217,7 @@ included the <a href="https://github.com/GoogleCloudPlatform/DataflowJavaSDK">Da
<!-- Render a "read more" button if the post is longer than the excerpt -->
<p>
-<a class="btn btn-default btn-sm" href="/beam/python/sdk/2016/02/25/python-sdk-now-public.html#read-more" role="button">
+<a class="btn btn-default btn-sm" href="/beam/python/sdk/2016/02/25/python-sdk-now-public.html" role="button">
Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"></span>
</a>
</p>
@@ -232,10 +232,13 @@ Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"><
both batch and stream processing into one powerful model. In fact, this unification
is so important, the name Beam itself comes from the union of <strong>B</strong>atch + str<strong>EAM</strong> = Beam</p>
+<p>When the project started, we wanted a logo which was both appealing and visually
+represented this unification.</p>
+
<!-- Render a "read more" button if the post is longer than the excerpt -->
<p>
-<a class="btn btn-default btn-sm" href="/beam/update/website/2016/02/22/beam-has-a-logo.html#read-more" role="button">
+<a class="btn btn-default btn-sm" href="/beam/update/website/2016/02/22/beam-has-a-logo.html" role="button">
Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"></span>
</a>
</p>