You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2019/05/31 17:12:36 UTC
[beam] branch asf-site updated: Publishing website 2019/05/31
17:12:25 at commit 2d3bcdc
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 2ca0293 Publishing website 2019/05/31 17:12:25 at commit 2d3bcdc
2ca0293 is described below
commit 2ca0293e4a060e644bf8e57bb8a4be3b5b10bb3c
Author: jenkins <bu...@apache.org>
AuthorDate: Fri May 31 17:12:25 2019 +0000
Publishing website 2019/05/31 17:12:25 at commit 2d3bcdc
---
.../documentation/execution-model/index.html | 59 +++++-
website/generated-content/documentation/index.html | 59 +++++-
.../io/built-in/google-bigquery/index.html | 59 +++++-
.../documentation/io/built-in/hadoop/index.html | 59 +++++-
.../documentation/io/built-in/hcatalog/index.html | 59 +++++-
.../documentation/io/built-in/index.html | 59 +++++-
.../documentation/io/developing-io-java/index.html | 59 +++++-
.../io/developing-io-overview/index.html | 59 +++++-
.../io/developing-io-python/index.html | 59 +++++-
.../documentation/io/testing/index.html | 59 +++++-
.../pipelines/create-your-pipeline/index.html | 59 +++++-
.../pipelines/design-your-pipeline/index.html | 59 +++++-
.../pipelines/test-your-pipeline/index.html | 59 +++++-
.../documentation/programming-guide/index.html | 59 +++++-
.../resources/learning-resources/index.html | 59 +++++-
.../resources/videos-and-podcasts/index.html | 59 +++++-
.../documentation/runners/jstorm/index.html | 59 +++++-
.../aggregation/approximatequantiles}/index.html | 133 ++++++------
.../aggregation/approximateunique}/index.html | 134 ++++++------
.../python/aggregation/cogroupbykey}/index.html | 152 ++++++++------
.../python/aggregation/combineglobally}/index.html | 144 +++++++------
.../aggregation/combinewithcontext}/index.html | 133 ++++++------
.../python/aggregation/count}/index.html | 147 +++++++------
.../python/aggregation/distinct}/index.html | 140 +++++++------
.../python/aggregation/groupbykey}/index.html | 142 +++++++------
.../aggregation/groupintobatches}/index.html | 133 ++++++------
.../python/aggregation/latest}/index.html | 133 ++++++------
.../python/aggregation/max}/index.html | 133 ++++++------
.../python/aggregation/mean}/index.html | 146 +++++++------
.../python/aggregation/min}/index.html | 133 ++++++------
.../python/aggregation/sample}/index.html | 147 +++++++------
.../python/aggregation/sum}/index.html | 133 ++++++------
.../python/aggregation/top}/index.html | 147 +++++++------
.../python/elementwise/filter}/index.html | 149 +++++++------
.../python/elementwise/flatmap}/index.html | 149 +++++++------
.../python/elementwise/keys}/index.html | 140 +++++++------
.../python/elementwise/kvswap}/index.html | 146 +++++++------
.../python/elementwise/map}/index.html | 145 +++++++------
.../python/elementwise/pardo}/index.html | 151 ++++++++------
.../python/elementwise/partition}/index.html | 149 +++++++------
.../python/elementwise/regex}/index.html | 133 ++++++------
.../python/elementwise/reify}/index.html | 133 ++++++------
.../python/elementwise/tostring}/index.html | 134 ++++++------
.../python/elementwise/values}/index.html | 140 +++++++------
.../python/elementwise/withkeys}/index.html | 134 ++++++------
.../python/elementwise/withtimestamps}/index.html | 133 ++++++------
.../python/other/create}/index.html | 149 +++++++------
.../python/other/flatten}/index.html | 144 +++++++------
.../python/other/passert}/index.html | 134 ++++++------
.../python/other/reshuffle}/index.html | 150 +++++++------
.../{ => transforms/python/other/view}/index.html | 134 ++++++------
.../python/other/windowinto}/index.html | 149 +++++++------
.../python/overview}/index.html | 231 +++++++++++----------
53 files changed, 3741 insertions(+), 2419 deletions(-)
diff --git a/website/generated-content/documentation/execution-model/index.html b/website/generated-content/documentation/execution-model/index.html
index 3d69522..587de2a 100644
--- a/website/generated-content/documentation/execution-model/index.html
+++ b/website/generated-content/documentation/execution-model/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/index.html
index 7ee0437..54f60ae 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/io/built-in/google-bigquery/index.html b/website/generated-content/documentation/io/built-in/google-bigquery/index.html
index ae093ba..fed1fab 100644
--- a/website/generated-content/documentation/io/built-in/google-bigquery/index.html
+++ b/website/generated-content/documentation/io/built-in/google-bigquery/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/io/built-in/hadoop/index.html b/website/generated-content/documentation/io/built-in/hadoop/index.html
index 1635a7c..ee96f01 100644
--- a/website/generated-content/documentation/io/built-in/hadoop/index.html
+++ b/website/generated-content/documentation/io/built-in/hadoop/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/io/built-in/hcatalog/index.html b/website/generated-content/documentation/io/built-in/hcatalog/index.html
index 58de197..b9c7ab9 100644
--- a/website/generated-content/documentation/io/built-in/hcatalog/index.html
+++ b/website/generated-content/documentation/io/built-in/hcatalog/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/io/built-in/index.html b/website/generated-content/documentation/io/built-in/index.html
index 68882ca..bc1f640 100644
--- a/website/generated-content/documentation/io/built-in/index.html
+++ b/website/generated-content/documentation/io/built-in/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/io/developing-io-java/index.html b/website/generated-content/documentation/io/developing-io-java/index.html
index 31d11de..13a0baa 100644
--- a/website/generated-content/documentation/io/developing-io-java/index.html
+++ b/website/generated-content/documentation/io/developing-io-java/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/io/developing-io-overview/index.html b/website/generated-content/documentation/io/developing-io-overview/index.html
index 33dfe8b..078d552 100644
--- a/website/generated-content/documentation/io/developing-io-overview/index.html
+++ b/website/generated-content/documentation/io/developing-io-overview/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/io/developing-io-python/index.html b/website/generated-content/documentation/io/developing-io-python/index.html
index a376179..acebc8b 100644
--- a/website/generated-content/documentation/io/developing-io-python/index.html
+++ b/website/generated-content/documentation/io/developing-io-python/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/io/testing/index.html b/website/generated-content/documentation/io/testing/index.html
index 57de05d..93111f3 100644
--- a/website/generated-content/documentation/io/testing/index.html
+++ b/website/generated-content/documentation/io/testing/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/pipelines/create-your-pipeline/index.html b/website/generated-content/documentation/pipelines/create-your-pipeline/index.html
index cc49afa..64e33ab 100644
--- a/website/generated-content/documentation/pipelines/create-your-pipeline/index.html
+++ b/website/generated-content/documentation/pipelines/create-your-pipeline/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/pipelines/design-your-pipeline/index.html b/website/generated-content/documentation/pipelines/design-your-pipeline/index.html
index bf83523..b40c062 100644
--- a/website/generated-content/documentation/pipelines/design-your-pipeline/index.html
+++ b/website/generated-content/documentation/pipelines/design-your-pipeline/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/pipelines/test-your-pipeline/index.html b/website/generated-content/documentation/pipelines/test-your-pipeline/index.html
index 0488f5d..a5dc57b 100644
--- a/website/generated-content/documentation/pipelines/test-your-pipeline/index.html
+++ b/website/generated-content/documentation/pipelines/test-your-pipeline/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/programming-guide/index.html b/website/generated-content/documentation/programming-guide/index.html
index f8b1a87..1b18588 100644
--- a/website/generated-content/documentation/programming-guide/index.html
+++ b/website/generated-content/documentation/programming-guide/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/resources/learning-resources/index.html b/website/generated-content/documentation/resources/learning-resources/index.html
index 02fb2fd..54c5772 100644
--- a/website/generated-content/documentation/resources/learning-resources/index.html
+++ b/website/generated-content/documentation/resources/learning-resources/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/resources/videos-and-podcasts/index.html b/website/generated-content/documentation/resources/videos-and-podcasts/index.html
index e0d1bb3..21a3228 100644
--- a/website/generated-content/documentation/resources/videos-and-podcasts/index.html
+++ b/website/generated-content/documentation/resources/videos-and-podcasts/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/runners/jstorm/index.html b/website/generated-content/documentation/runners/jstorm/index.html
index cce9b65..b3dd91d 100644
--- a/website/generated-content/documentation/runners/jstorm/index.html
+++ b/website/generated-content/documentation/runners/jstorm/index.html
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/approximatequantiles/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/approximatequantiles/index.html
index 7ee0437..bd94616 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/approximatequantiles/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>ApproximateQuantiles</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/approximatequantiles/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/approximatequantiles.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,61 +369,12 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
+<h1 id="approximatequantiles">ApproximateQuantiles</h1>
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
+<h2 id="related-transforms">Related transforms</h2>
</div>
</div>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/approximateunique/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/approximateunique/index.html
index 7ee0437..8ecc1e2 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/approximateunique/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>ApproximateUnique</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/approximateunique/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/approximateunique.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -318,62 +368,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
+<h1 id="approximateunique">ApproximateUnique</h1>
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
+<h2 id="related-transforms">Related transforms</h2>
</div>
</div>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/cogroupbykey/index.html
similarity index 79%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/cogroupbykey/index.html
index 7ee0437..fb15e23 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/cogroupbykey/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>CoGroupByKey</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/cogroupbykey/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/cogroupbykey.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,32 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
+<h1 id="cogroupbykey">CoGroupByKey</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.util.html#apache_beam.transforms.util.CoGroupByKey">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Aggregates all input elements by their key and allows downstream processing
+to consume all values associated with the key. While <code class="highlighter-rouge">GroupByKey</code> performs
+this operation over a single input collection and thus a single type of input
+values, <code class="highlighter-rouge">CoGroupByKey</code> operates over multiple input collections. As a result,
+the result for each key is a tuple of the values associated with that key in
+each input collection.</p>
+
+<p>See more information in the <a href="/documentation/programming-guide/#cogroupbykey">Beam Programming Guide</a>.</p>
+
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
+
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally">CombineGlobally</a> to combine elements.</li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey">GroupByKey</a> takes one input collection.</li>
</ul>
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/combineglobally/index.html
similarity index 80%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/combineglobally/index.html
index 7ee0437..5adf05f 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/combineglobally/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>CombineGlobally</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/combineglobally/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/combineglobally.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -318,63 +368,31 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
+<h1 id="combineglobally">CombineGlobally</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.CombineGlobally">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br /></p>
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
+<p>Combines all elements in a collection.</p>
-<h2 id="concepts">Concepts</h2>
+<p>See more information in the <a href="/documentation/programming-guide/#combine">Beam Programming Guide</a>.</p>
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
+<h2 id="related-transforms">Related transforms</h2>
+<p>You can use the following combiner transforms:</p>
<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
+ <li><a href="/documentation/transforms/python/aggregation/mean">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top">Top</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample">Sample</a></li>
</ul>
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/combinewithcontext/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/combinewithcontext/index.html
index 7ee0437..374547c 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/combinewithcontext/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>CombineWithContext</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/combinewithcontext/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/combinewithcontext.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,61 +369,12 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
+<h1 id="combinewithcontext">CombineWithContext</h1>
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
+<h2 id="related-transforms">Related transforms</h2>
</div>
</div>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/count/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/count/index.html
index 7ee0437..553219e 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/count/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Count</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/count/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/count.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -318,62 +368,21 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
+<h1 id="count">Count</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/2.12.0/apache_beam.transforms.combiners.html?#apache_beam.transforms.combiners.Count">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Counts the number of elements within each aggregation.</p>
+
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
+
+<h2 id="related-transforms">Related transforms</h2>
+<p>N/A</p>
</div>
</div>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/distinct/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/distinct/index.html
index 7ee0437..f0de107 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/distinct/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Distinct</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/distinct/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/distinct.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,24 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
+<h1 id="distinct">Distinct</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.util.html#apache_beam.transforms.util.Distinct">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Produces a collection containing distinct elements of the input collection.</p>
-<h3 id="available-runners">Available Runners</h3>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
+ <li><a href="/documentation/transforms/python/aggregation/count">Count</a> counts the number of elements within each aggregation.</li>
</ul>
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/groupbykey/index.html
similarity index 80%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/groupbykey/index.html
index 7ee0437..cb41768 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/groupbykey/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>GroupByKey</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/groupbykey/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/groupbykey.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,28 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
+<h1 id="groupbykey">GroupByKey</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.GroupByKey">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Takes a keyed collection of elements and produces a collection
+where each element consists of a key and all values associated with that key.</p>
-<h2 id="concepts">Concepts</h2>
+<p>See more information in the <a href="/documentation/programming-guide/#groupbykey">Beam Programming Guide</a>.</p>
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally">CombineGlobally</a> for combining all values associated with a key to a single result.</li>
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey">CoGroupByKey</a> for multiple input collections.</li>
</ul>
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/groupintobatches/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/groupintobatches/index.html
index 7ee0437..e2a5e1b 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/groupintobatches/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>GroupIntoBatches</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/groupintobatches/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/groupintobatches.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,61 +369,12 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
+<h1 id="groupintobatches">GroupIntoBatches</h1>
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
+<h2 id="related-transforms">Related transforms</h2>
</div>
</div>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/latest/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/latest/index.html
index 7ee0437..b1cd04a 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/latest/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Latest</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/latest/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/latest.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -318,63 +368,16 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
+<h1 id="latest">Latest</h1>
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
+ <li><a href="/documentation/transforms/python/aggregation/sample">Sample</a> to combine elements. takes samples of the elements in a collection.</li>
</ul>
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/max/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/max/index.html
index 7ee0437..9ca7fdf 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/max/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Max</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/max/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/max.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,61 +369,12 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
+<h1 id="max">Max</h1>
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
+<h2 id="related-transforms">Related transforms</h2>
</div>
</div>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/mean/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/mean/index.html
index 7ee0437..4059a36 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/mean/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Mean</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/mean/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/mean.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,26 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
+<h1 id="mean">Mean</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.combiners.html?highlight=mean#apache_beam.transforms.combiners.Mean">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Transforms for computing the arithmetic mean of the elements in a collection,
+or the mean of the values associated with each key in a collection of
+key-value pairs.</p>
+
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
+
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally">CombineGlobally</a> to combine elements.</li>
</ul>
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/min/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/min/index.html
index 7ee0437..1271578 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/min/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Min</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/min/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/min.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,61 +369,12 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
+<h1 id="min">Min</h1>
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
+<h2 id="related-transforms">Related transforms</h2>
</div>
</div>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/sample/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/sample/index.html
index 7ee0437..eaba924 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/sample/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Sample</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/sample/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/sample.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -318,63 +368,26 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
+<h1 id="sample">Sample</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/2.12.0/apache_beam.transforms.combiners.html?#apache_beam.transforms.combiners.Sample">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Transforms for taking samples of the elements in a collection, or
+samples of the values associated with each key in a collection of
+key-value pairs.</p>
+
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
+
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
+ <li><a href="/documentation/transforms/python/aggregation/top">Top</a> finds the largest or smallest element.</li>
</ul>
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/sum/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/sum/index.html
index 7ee0437..acd0461 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/sum/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Sum</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/sum/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/sum.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,61 +369,12 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
+<h1 id="sum">Sum</h1>
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
+<h2 id="related-transforms">Related transforms</h2>
</div>
</div>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/aggregation/top/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/aggregation/top/index.html
index 7ee0437..782d6b9 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/top/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Top</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/aggregation/top/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/aggregation/top.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -318,63 +368,26 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
+<h1 id="top">Top</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/2.12.0/apache_beam.transforms.combiners.html?#apache_beam.transforms.combiners.Top">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Transforms for finding the largest (or smallest) set of elements in
+a collection, or the largest (or smallest) set of values associated
+with each key in a collection of key-value pairs.</p>
+
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7390">BEAM-7390</a> for updates.</p>
+
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
+ <li><a href="/documentation/transforms/python/aggregation/sample">Sample</a> to combine elements. takes samples of the elements in a collection.</li>
</ul>
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/elementwise/filter/index.html
similarity index 80%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/elementwise/filter/index.html
index 7ee0437..3280ac2 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/filter/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Filter</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/elementwise/filter/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/element-wise/filter.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,29 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
+<h1 id="filter">Filter</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.Filter">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Given a predicate, filter out all elements that don’t satisfy that predicate.
+May also be used to filter based on an inequality with a given value based
+on the comparison ordering of the element.</p>
+
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7389">BEAM-7389</a> for updates.</p>
+
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap">FlatMap</a> behaves the same as <code class="highlighter-rouge">Map</code>, but for
+each input it might produce zero or more outputs.</li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo">ParDo</a> is the most general element-wise mapping
+operation, and includes other abilities such as multiple output collections and side-inputs.</li>
</ul>
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/elementwise/flatmap/index.html
similarity index 80%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/elementwise/flatmap/index.html
index 7ee0437..ad34689 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/flatmap/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>FlatMap</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/elementwise/flatmap/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/element-wise/flatmap.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,29 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
+<h1 id="flatmap">FlatMap</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.FlatMap">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Applies a simple 1-to-many mapping function over each element in the collection.
+The many elements are flattened into the resulting collection.</p>
+
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7389">BEAM-7389</a> for updates.</p>
+
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/filter">Filter</a> is useful if the function is just
+deciding whether to output an element or not.</li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo">ParDo</a> is the most general element-wise mapping
+operation, and includes other abilities such as multiple output collections and side-inputs.</li>
+ <li><a href="/documentation/transforms/python/elementwise/map">Map</a> behaves the same, but produces exactly one output for each input.</li>
</ul>
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/elementwise/keys/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/elementwise/keys/index.html
index 7ee0437..6ac3e7a 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/keys/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Keys</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/elementwise/keys/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/element-wise/keys.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,61 +369,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
+<h1 id="keys">Keys</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.util.html#apache_beam.transforms.util.Keys">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Takes a collection of key-value pairs and returns the key of each element.</p>
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7389">BEAM-7389</a> for updates.</p>
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap">KvSwap</a> swaps the key and value of each element.</li>
+ <li><a href="/documentation/transforms/python/elementwise/values">Values</a> for extracting the value of each element.</li>
</ul>
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
</div>
</div>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/elementwise/kvswap/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/elementwise/kvswap/index.html
index 7ee0437..0a5cda0 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/kvswap/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>KvSwap</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/elementwise/kvswap/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/element-wise/kvswap.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,26 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
+<h1 id="kvswap">Kvswap</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.util.html#apache_beam.transforms.util.KvSwap">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Takes a collection of key-value pairs and returns a collection of key-value pairs
+which has each key and value swapped.</p>
+
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7389">BEAM-7389</a> for updates.</p>
+
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys">Keys</a> for extracting the key of each component.</li>
+ <li><a href="/documentation/transforms/python/elementwise/values">Values</a> for extracting the value of each element.</li>
</ul>
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/elementwise/map/index.html
similarity index 80%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/elementwise/map/index.html
index 7ee0437..489818d 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/map/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Map</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/elementwise/map/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/element-wise/map.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,29 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
+<h1 id="map">Map</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.Map">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Applies a simple 1-to-1 mapping function over each element in the collection.</p>
-<h3 id="available-runners">Available Runners</h3>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7389">BEAM-7389</a> for updates.</p>
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap">FlatMap</a> behaves the same as <code class="highlighter-rouge">Map</code>, but for
+each input it may produce zero or more outputs.</li>
+ <li><a href="/documentation/transforms/python/elementwise/filter">Filter</a> is useful if the function is just
+deciding whether to output an element or not.</li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo">ParDo</a> is the most general element-wise mapping
+operation, and includes other abilities such as multiple output collections and side-inputs.</li>
</ul>
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/elementwise/pardo/index.html
similarity index 80%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/elementwise/pardo/index.html
index 7ee0437..350a87d 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/pardo/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>ParDo</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/elementwise/pardo/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/element-wise/pardo.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,31 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
+<h1 id="pardo">ParDo</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.ParDo">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+A transform for generic parallel processing. A <code class="highlighter-rouge">ParDo</code> transform considers each
+element in the input <code class="highlighter-rouge">PCollection</code>, performs some processing function
+(your user code) on that element, and emits zero or more elements to
+an output PCollection.</p>
+
+<p>See more information in the <a href="/documentation/programming-guide/#pardo">Beam Programming Guide</a>.</p>
+
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7389">BEAM-7389</a> for updates.</p>
+
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap">FlatMap</a> behaves the same as <code class="highlighter-rouge">Map</code>, but for each input it may produce zero or more outputs.</li>
+ <li><a href="/documentation/transforms/python/elementwise/filter">Filter</a> is useful if the function is just
+deciding whether to output an element or not.</li>
</ul>
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/elementwise/partition/index.html
similarity index 79%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/elementwise/partition/index.html
index 7ee0437..df7380b 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/partition/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Partition</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/elementwise/partition/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/element-wise/partition.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,37 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
+<h1 id="partition">Partition</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.Partition">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Separates elements in a collection into multiple output
+collections. The partitioning function contains the logic that determines how
+to separate the elements of the input collection into each resulting
+partition output collection.</p>
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
+<p>The number of partitions must be determined at graph construction time.
+You cannot determine the number of partitions in mid-pipeline</p>
-<h2 id="sdks">SDKs</h2>
+<p>See more information in the <a href="/documentation/programming-guide/#partition">Beam Programming Guide</a>.</p>
-<p>Find status and reference information on all of the available Beam SDKs.</p>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7389">BEAM-7389</a> for updates.</p>
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/filter">Filter</a> is useful if the function is just
+deciding whether to output an element or not.</li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo">ParDo</a> is the most general element-wise mapping
+operation, and includes other abilities such as multiple output collections and side-inputs.</li>
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey">CoGroupByKey</a>
+performs a per-key equijoin.</li>
</ul>
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/elementwise/regex/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/elementwise/regex/index.html
index 7ee0437..00e2125 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/regex/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Regex</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/elementwise/regex/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/element-wise/regex.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,17 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
+<h1 id="regex">Regex</h1>
+<p>Filters input string elements based on a regex. May also transform them based on the matching groups.</p>
-<h3 id="available-runners">Available Runners</h3>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7389">BEAM-7389</a> for updates.</p>
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
+ <li><a href="/documentation/transforms/python/elementwise/map">Map</a> applies a simple 1-to-1 mapping function over each element in the collection</li>
</ul>
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/elementwise/reify/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/elementwise/reify/index.html
index 7ee0437..37da776 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/reify/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Reify</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/elementwise/reify/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/element-wise/reify.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,17 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
+<h1 id="reify">Reify</h1>
+<p>Transforms for converting between explicit and implicit form of various Beam values.</p>
-<h3 id="available-runners">Available Runners</h3>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7389">BEAM-7389</a> for updates.</p>
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps">WithTimestamps</a> assigns timestamps to all the elements of a collection.</li>
</ul>
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/elementwise/tostring/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/elementwise/tostring/index.html
index 7ee0437..5cee866 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/tostring/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>ToString</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/elementwise/tostring/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/element-wise/tostring.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,61 +369,13 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
+<h1 id="tostring">ToString</h1>
+<p>Transforms every element in an input collection a string.</p>
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7389">BEAM-7389</a> for updates.</p>
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
+<h2 id="related-transforms">Related transforms</h2>
</div>
</div>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/elementwise/values/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/elementwise/values/index.html
index 7ee0437..7e92f25 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/values/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Values</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/elementwise/values/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/element-wise/values.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,24 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
+<h1 id="values">Values</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.util.html#apache_beam.transforms.util.Values">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Takes a collection of key-value pairs, and returns the value of each element.</p>
-<h3 id="available-runners">Available Runners</h3>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7389">BEAM-7389</a> for updates.</p>
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
+ <li><a href="/documentation/transforms/python/elementwise/keys">Keys</a> for extracting the key of each component.</li>
</ul>
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/elementwise/withkeys/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/elementwise/withkeys/index.html
index 7ee0437..f4c63fc 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/withkeys/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>WithKeys</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/elementwise/withkeys/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/element-wise/withkeys.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,61 +369,13 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
+<h1 id="withkeys">WithKeys</h1>
+<p>Not available.</p>
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7389">BEAM-7389</a> for updates.</p>
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
+<h2 id="related-transforms">Related transforms</h2>
</div>
</div>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/elementwise/withtimestamps/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/elementwise/withtimestamps/index.html
index 7ee0437..e870448 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/withtimestamps/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>WithTimestamps</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/elementwise/withtimestamps/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/element-wise/withtimestamps.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,17 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
+<h1 id="withtimestamps">WithTimestamps</h1>
+<p>Assigns timestamps to all the elements of a collection.</p>
-<h3 id="available-runners">Available Runners</h3>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7389">BEAM-7389</a> for updates.</p>
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
+ <li><a href="/documentation/transforms/python/elementwise/reify">Reify</a> converts between explicit and implicit forms of Beam values.</li>
</ul>
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/other/create/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/other/create/index.html
index 7ee0437..c09b239 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/other/create/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Create</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/other/create/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/other/create.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -318,62 +368,23 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
+<h1 id="create">Create</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.Create">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Creates a collection containing a specified set of elements. This is
+useful for testing, as well as creating an initial input to process
+in parallel. For example, a single element to execute a one-time
+<code class="highlighter-rouge">ParDo</code> or a list of filenames to be read.</p>
+
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7391">BEAM-7391</a> for updates.</p>
+
+<h2 id="related-transforms">Related transforms</h2>
</div>
</div>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/other/flatten/index.html
similarity index 80%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/other/flatten/index.html
index 7ee0437..4088751 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/other/flatten/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Flatten</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/other/flatten/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/other/flatten.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,30 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
+<h1 id="flatten">Flatten</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html?highlight=flatten#apache_beam.transforms.core.Flatten">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Merges multiple <code class="highlighter-rouge">PCollection</code> objects into a single logical
+<code class="highlighter-rouge">PCollection</code>. A transform for <code class="highlighter-rouge">PCollection</code> objects
+that store the same data type.</p>
-<h2 id="concepts">Concepts</h2>
+<p>See more information in the <a href="/documentation/programming-guide/#flatten">Beam Programming Guide</a>.</p>
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7391">BEAM-7391</a> for updates.</p>
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap">FlatMap</a> applies a simple 1-to-many mapping
+function over each element in the collection. This transform might produce zero
+or more outputs.</li>
</ul>
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/other/passert/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/other/passert/index.html
index 7ee0437..7bd1997 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/other/passert/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>PAssert</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/other/passert/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/other/passert.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -318,62 +368,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
+<h1 id="passert">PAssert</h1>
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7391">BEAM-7391</a> for updates.</p>
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
+<h2 id="related-transforms">Related transforms</h2>
</div>
</div>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/other/reshuffle/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/other/reshuffle/index.html
index 7ee0437..c49be06 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/other/reshuffle/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>Reshuffle</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/other/reshuffle/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/other/reshuffle.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,61 +369,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
+<h1 id="reshuffle">Reshuffle</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.util.html?highlight=reshuffle#apache_beam.transforms.util.Reshuffle">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+ Adds a temporary random key to each element in a collection, reshuffles
+ these keys, and removes the temporary key. This redistributes the
+ elements between workers and returns a collection equivalent to its
+ input collection. This is most useful for adjusting paralellism or
+ preventing coupled failures.</p>
+
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7391">BEAM-7391</a> for updates.</p>
+
+<h2 id="related-transforms">Related transforms</h2>
+<p>N/A</p>
</div>
</div>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/other/view/index.html
similarity index 81%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/other/view/index.html
index 7ee0437..6adb455 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/other/view/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>View</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/other/view/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/other/view.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -318,62 +368,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
+<h1 id="view">View</h1>
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
-<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
-</ul>
-
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7391">BEAM-7391</a> for updates.</p>
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
+<h2 id="related-transforms">Related transforms</h2>
</div>
</div>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/transforms/python/other/windowinto/index.html
similarity index 80%
copy from website/generated-content/documentation/index.html
copy to website/generated-content/documentation/transforms/python/other/windowinto/index.html
index 7ee0437..a7ccfe0 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/transforms/python/other/windowinto/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Learn about Beam</title>
+ <title>WindowInto</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/other/windowinto/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/index.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/other/windowinto.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,15 +347,8 @@
<ul class="nav">
- <li><a href="#concepts">Concepts</a></li>
- <li><a href="#pipeline-fundamentals">Pipeline Fundamentals</a></li>
- <li><a href="#sdks">SDKs</a></li>
- <li><a href="#runners">Runners</a>
- <ul>
- <li><a href="#available-runners">Available Runners</a></li>
- <li><a href="#choosing-a-runner">Choosing a Runner</a></li>
- </ul>
- </li>
+ <li><a href="#examples">Examples</a></li>
+ <li><a href="#related-transforms">Related transforms</a></li>
</ul>
@@ -319,62 +369,29 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-documentation">Apache Beam Documentation</h1>
-
-<p>This section provides in-depth conceptual information and reference material for the Beam Model, SDKs, and Runners:</p>
-
-<h2 id="concepts">Concepts</h2>
-
-<p>Learn about the Beam Programming Model and the concepts common to all Beam SDKs and Runners.</p>
-
-<ul>
- <li>The <a href="/documentation/programming-guide/">Programming Guide</a> introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
- <li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
-</ul>
-
-<h2 id="pipeline-fundamentals">Pipeline Fundamentals</h2>
-
-<ul>
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a> by planning your pipeline’s structure, choosing transforms to apply to your data, and determining your input and output methods.</li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a> using the classes in the Beam SDKs.</li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a> to minimize debugging a pipeline’s remote execution.</li>
-</ul>
-
-<h2 id="sdks">SDKs</h2>
-
-<p>Find status and reference information on all of the available Beam SDKs.</p>
-
+<h1 id="windowinto">WindowInto</h1>
+<table align="left">
+ <a target="_blank" class="button" href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.window.html?highlight=window#module-apache_beam.transforms.window">
+ <img src="https://beam.apache.org/images/logos/sdks/python.png" width="20px" height="20px" alt="Pydoc" />
+ Pydoc
+ </a>
+</table>
+<p><br />
+Logically divides up or groups the elements of a collection into finite
+windows according to a function.</p>
+
+<h2 id="examples">Examples</h2>
+<p>See <a href="https://issues.apache.org/jira/browse/BEAM-7391">BEAM-7391</a> for updates.</p>
+
+<h2 id="related-transforms">Related transforms</h2>
<ul>
- <li><a href="/documentation/sdks/java/">Java SDK</a></li>
- <li><a href="/documentation/sdks/python/">Python SDK</a></li>
- <li><a href="/documentation/sdks/go/">Go SDK</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey">GroupByKey</a>
+produces a collection where each element consists of a key and all values associated
+with that key.</li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps">Timestamp</a>
+applies a function to determine a timestamp to each element in the output collection.</li>
</ul>
-<h2 id="runners">Runners</h2>
-
-<p>A Beam Runner runs a Beam pipeline on a specific (often distributed) data processing system.</p>
-
-<h3 id="available-runners">Available Runners</h3>
-
-<ul>
- <li><a href="/documentation/runners/direct/">DirectRunner</a>: Runs locally on your machine – great for developing, testing, and debugging.</li>
- <li><a href="/documentation/runners/apex/">ApexRunner</a>: Runs on <a href="http://apex.apache.org">Apache Apex</a>.</li>
- <li><a href="/documentation/runners/flink/">FlinkRunner</a>: Runs on <a href="http://flink.apache.org">Apache Flink</a>.</li>
- <li><a href="/documentation/runners/spark/">SparkRunner</a>: Runs on <a href="http://spark.apache.org">Apache Spark</a>.</li>
- <li><a href="/documentation/runners/dataflow/">DataflowRunner</a>: Runs on <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>, a fully managed service within <a href="https://cloud.google.com/">Google Cloud Platform</a>.</li>
- <li><a href="/documentation/runners/gearpump/">GearpumpRunner</a>: Runs on <a href="http://gearpump.apache.org">Apache Gearpump (incubating)</a>.</li>
- <li><a href="/documentation/runners/samza/">SamzaRunner</a>: Runs on <a href="http://samza.apache.org">Apache Samza</a>.</li>
- <li><a href="/documentation/runners/nemo/">NemoRunner</a>: Runs on <a href="http://nemo.apache.org">Apache Nemo</a>.</li>
- <li><a href="/documentation/runners/jet/">JetRunner</a>: Runs on <a href="https://jet.hazelcast.org/">Hazelcast Jet</a>.</li>
-</ul>
-
-<h3 id="choosing-a-runner">Choosing a Runner</h3>
-
-<p>Beam is designed to enable pipelines to be portable across different runners. However, given every runner has different capabilities, they also have different abilities to implement the core concepts in the Beam model. The <a href="/documentation/runners/capability-matrix/">Capability Matrix</a> provides a detailed comparison of runner functionality.</p>
-
-<p>Once you have chosen which runner to use, see that runner’s page for more information about any initial runner-specific setup as well as any required or optional <code class="highlighter-rouge">PipelineOptions</code> for configuring it’s execution. You may also want to refer back to the Quickstart for <a href="/get-started/quickstart-java">Java</a>, <a href="/get-started/quickstart-py">Python</a> or <a href="/get-started/quickstart-go">Go</a> for instructions on executing the sample W [...]
-
</div>
</div>
<!--
diff --git a/website/generated-content/documentation/runners/jstorm/index.html b/website/generated-content/documentation/transforms/python/overview/index.html
similarity index 67%
copy from website/generated-content/documentation/runners/jstorm/index.html
copy to website/generated-content/documentation/transforms/python/overview/index.html
index cce9b65..f2d583b 100644
--- a/website/generated-content/documentation/runners/jstorm/index.html
+++ b/website/generated-content/documentation/transforms/python/overview/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>JStorm Runner</title>
+ <title>Python transform catalog overview</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/runners/jstorm/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/transforms/python/overview/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -125,7 +125,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/runners/jstorm.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/transforms/python/index.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -254,8 +254,65 @@
</li>
</ul>
</li>
+
+<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transform catalog</span>
+
+ <ul class="section-nav-list">
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Python</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Element-wise</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
+ <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Aggregation</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
+ <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Other</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
+ <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
+ <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
+ <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+</li>
+
<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning Resources</span>
+ <span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
<li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
@@ -290,17 +347,9 @@
<ul class="nav">
- <li><a href="#jstorm-runner-prerequisites-and-setup">JStorm Runner prerequisites and setup</a>
- <ul>
- <li><a href="#deploying-jstorm-with-your-application">Deploying JStorm with your application</a></li>
- </ul>
- </li>
- <li><a href="#pipeline-options-for-the-jstorm-runner">Pipeline options for the JStorm Runner</a></li>
- <li><a href="#additional-notes">Additional notes</a>
- <ul>
- <li><a href="#monitoring-your-job">Monitoring your job</a></li>
- </ul>
- </li>
+ <li><a href="#element-wise">Element-wise</a></li>
+ <li><a href="#aggregation">Aggregation</a></li>
+ <li><a href="#other">Other</a></li>
</ul>
@@ -320,104 +369,70 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="using-the-jstorm-runner">Using the JStorm Runner</h1>
-
-<p>The JStorm Runner can be used to execute Beam pipelines using <a href="http://jstorm.io/">JStorm</a>, while providing:</p>
-<ul>
- <li>High throughput and low latency.</li>
- <li>At-least-once and exactly-once fault tolerance.</li>
-</ul>
+<h1 id="python-transform-catalog-overview">Python transform catalog overview</h1>
+
+<h2 id="element-wise">Element-wise</h2>
+
+<table class="table-bordered table-striped">
+ <tr><th>Transform</th><th>Description</th></tr>
+ <tr><td><a href="/documentation/transforms/python/elementwise/filter">Filter</a></td><td>Given a predicate, filter out all elements that don't satisfy the predicate.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/elementwise/flatmap">FlatMap</a></td><td>Applies a function that returns a collection to every element in the input and
+ outputs all resulting elements.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/elementwise/keys">Keys</a></td><td>Extracts the key from each element in a collection of key-value pairs.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/elementwise/kvswap">KvSwap</a></td><td>Swaps the key and value of each element in a collection of key-value pairs.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/elementwise/map">Map</a></td><td>Applies a function to every element in the input and outputs the result.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/elementwise/pardo">ParDo</a></td><td>The most-general mechanism for applying a user-defined <code>DoFn</code> to every element
+ in the input collection.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/elementwise/partition">Partition</a></td><td>Routes each input element to a specific output collection based on some partition
+ function.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/elementwise/regex">Regex</a></td><td>Filters input string elements based on a regex. May also transform them based on the matching groups.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/elementwise/reify">Reify</a></td><td>Transforms for converting between explicit and implicit form of various Beam values.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/elementwise/tostring">ToString</a></td><td>Transforms every element in an input collection a string.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/elementwise/withtimestamps">WithTimestamps</a></td><td>Applies a function to determine a timestamp to each element in the output collection,
+ and updates the implicit timestamp associated with each input. Note that it is only
+ safe to adjust timestamps forwards.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/elementwise/values">Values</a></td><td>Extracts the value from each element in a collection of key-value pairs.</td></tr>
+</table>
-<p>Like a native JStorm topology, users can execute Beam topology with local mode, standalone cluster or jstorm-on-yarn cluster.</p>
-
-<p>The <a href="/documentation/runners/capability-matrix/">Beam Capability Matrix</a> documents the currently supported capabilities of the JStorm Runner.</p>
-
-<h2 id="jstorm-runner-prerequisites-and-setup">JStorm Runner prerequisites and setup</h2>
-
-<p>The JStorm runner currently supports JStorm version 2.5.0-SNAPSHOT.</p>
-
-<p>You can add a dependency on the latest version of the JStorm runner by adding the following to your pom.xml:</p>
-<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="o"><</span><span class="n">dependency</span><span class="o">></span>
- <span class="o"><</span><span class="n">groupId</span><span class="o">></span><span class="n">org</span><span class="o">.</span><span class="na">apache</span><span class="o">.</span><span class="na">beam</span><span class="o"></</span><span class="n">groupId</span><span class="o">></span>
- <span class="o"><</span><span class="n">artifactId</span><span class="o">></span><span class="n">beam</span><span class="o">-</span><span class="n">runners</span><span class="o">-</span><span class="n">jstorm</span><span class="o"></</span><span class="n">artifactId</span><span class="o">></span>
- <span class="o"><</span><span class="n">version</span><span class="o">></span><span class="mf">2.12</span><span class="o">.</span><span class="mi">0</span><span class="o"></</span><span class="n">version</span><span class="o">></span>
-<span class="o"></</span><span class="n">dependency</span><span class="o">></span>
-</code></pre>
-</div>
-
-<h3 id="deploying-jstorm-with-your-application">Deploying JStorm with your application</h3>
-
-<p>To run against a Standalone cluster, you can package your program with all Beam dependencies into a fat jar, and then submit the topology with the following command.</p>
-<div class="highlighter-rouge"><pre class="highlight"><code>jstorm jar WordCount.jar org.apache.beam.examples.WordCount --runner=org.apache.beam.runners.jstorm.JStormRunner
-</code></pre>
-</div>
-
-<p>If you don’t want to package a fat jar, you can upload the Beam dependencies onto all cluster nodes(<code class="highlighter-rouge">$JSTORM_HOME/lib/ext/beam</code>) first.
-When you submit a topology with argument <code class="highlighter-rouge">"--external-libs beam"</code>, JStorm will load the Beam dependencies automatically.</p>
-<div class="highlighter-rouge"><pre class="highlight"><code>jstorm jar WordCount.jar org.apache.beam.examples.WordCount --external-libs beam --runner=org.apache.beam.runners.jstorm.JStormRunner
-</code></pre>
-</div>
-
-<p>To learn about deploying a JStorm cluster, please refer to <a href="http://jstorm.io/QuickStart/Deploy/index.html">JStorm cluster deploy</a></p>
-
-<h2 id="pipeline-options-for-the-jstorm-runner">Pipeline options for the JStorm Runner</h2>
-
-<p>When executing your pipeline with the JStorm Runner, you should consider the following pipeline options.</p>
-
-<table class="table table-bordered">
-<tr>
- <th>Field</th>
- <th>Description</th>
- <th>Default Value</th>
-</tr>
-<tr>
- <td><code>runner</code></td>
- <td>The pipeline runner to use. This option allows you to determine the pipeline runner at runtime.</td>
- <td>Set to <code>JStormRunner</code> to run using JStorm.</td>
-</tr>
-<tr>
- <td><code>topologyConfig</code></td>
- <td>System topology config of JStorm</td>
- <td>DefaultMapValueFactory.class</td>
-</tr>
-<tr>
- <td><code>workerNumber</code></td>
- <td>Worker number of topology</td>
- <td>1</td>
-</tr>
-<tr>
- <td><code>parallelism</code></td>
- <td>Global parallelism number of a component</td>
- <td>1</td>
-</tr>
-<tr>
- <td><code>parallelismMap</code></td>
- <td>Parallelism number of a specified composite PTransform</td>
- <td>DefaultMapValueFactory.class</td>
-</tr>
-<tr>
- <td><code>exactlyOnceTopology</code></td>
- <td>Indicate if it is an exactly once topology</td>
- <td>false</td>
-</tr>
-<tr>
- <td><code>localMode</code></td>
- <td>Indicate if the topology is running on local machine or distributed cluster</td>
- <td>false</td>
-</tr>
-<tr>
- <td><code>localModeExecuteTimeSec</code></td>
- <td>Executing time(sec) of topology on local mode.</td>
- <td>60</td>
-</tr>
+<h2 id="aggregation">Aggregation</h2>
+<table class="table-bordered table-striped">
+ <tr><th>Transform</th><th>Description</th></tr>
+ <tr><td>ApproximateQuantiles</td><td>Not available. See <a href="https://issues.apache.org/jira/browse/BEAM-6694">BEAM-6694</a> for updates.</td></tr>
+ <tr><td>ApproximateUnique</td><td>Not available. See <a href="https://issues.apache.org/jira/browse/BEAM-6693">BEAM-6693</a> for updates.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/aggregation/cogroupbykey">CoGroupByKey</a></td><td>Takes several keyed collections of elements and produces a collection where each element
+ consists of a key and all values associated with that key.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/aggregation/combineglobally">CombineGlobally</a></td><td>Transforms to combine elements.</td></tr>
+ <tr><td>CombineWithContext</td><td>Not available.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/aggregation/count">Count</a></td><td>Counts the number of elements within each aggregation.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/aggregation/distinct">Distinct</a></td><td>Produces a collection containing distinct elements from the input collection.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/aggregation/groupbykey">GroupByKey</a></td><td>Takes a keyed collection of elements and produces a collection where each element
+ consists of a key and all values associated with that key.</td></tr>
+ <tr><td>GroupIntoBatches</td><td>Not available. See <a href="https://issues.apache.org/jira/browse/BEAM-6696">BEAM-6696</a> for updates.</td></tr>
+ <tr><td>Latest</td><td>Not available. See <a href="https://issues.apache.org/jira/browse/BEAM-6695">BEAM-6695</a> for updates.</td></tr>
+ <tr><td>Max</td><td>Not available.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/aggregation/mean">Mean</a></td><td>Computes the average within each aggregation.</td></tr>
+ <tr><td>Min</td><td>Not available.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/aggregation/sample">Sample</a></td><td>Randomly select some number of elements from each aggregation.</td></tr>
+ <tr><td>Sum</td><td>Not available.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/aggregation/top">Top</a></td><td>Compute the largest element(s) in each aggregation.</td></tr>
</table>
-<h2 id="additional-notes">Additional notes</h2>
+<h2 id="other">Other</h2>
+<table class="table-bordered table-striped">
+ <tr><th>Transform</th><th>Description</th></tr>
+ <tr><td><a href="/documentation/transforms/python/other/create">Create</a></td><td>Creates a collection from an in-memory list.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/other/flatten">Flatten</a></td><td>Given multiple input collections, produces a single output collection containing
+ all elements from all of the input collections.
+</td></tr>
+ <tr><td>PAssert</td><td>Not available.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/other/reshuffle">Reshuffle</a></td><td>Given an input collection, redistributes the elements between workers. This is
+ most useful for adjusting paralellism or preventing coupled failures.</td></tr>
+ <tr><td>View</td><td>Not available.</td></tr>
+ <tr><td><a href="/documentation/transforms/python/other/windowinto">WindowInto</a></td><td>Logically divides up or groups the elements of a collection into finite
+ windows according to a function.</td></tr>
+</table>
-<h3 id="monitoring-your-job">Monitoring your job</h3>
-<p>You can monitor your job with the JStorm UI, which displays all JStorm system metrics and Beam metrics.
-For testing on local mode, you can retreive the Beam metrics with the metrics method of PipelineResult.</p>
</div>
</div>