You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by al...@apache.org on 2019/09/26 00:59:31 UTC
[beam] branch master updated: [BEAM-7389] Use includes for buttons
This is an automated email from the ASF dual-hosted git repository.
altay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 97c4381 [BEAM-7389] Use includes for buttons
new 7120ceb Merge pull request #9661 from davidcavazos/include-buttons
97c4381 is described below
commit 97c4381adb2330467425395a88a97d03ec378670
Author: David Cavazos <dc...@google.com>
AuthorDate: Tue Sep 24 21:40:21 2019 -0700
[BEAM-7389] Use includes for buttons
---
.../transforms/python/element-wise/filter-py.ipynb | 50 ++--
.../python/element-wise/flatmap-py.ipynb | 70 ++----
website/notebooks/generate.py | 80 ++++--
website/src/_includes/button-pydoc.md | 23 ++
website/src/_includes/button.md | 21 ++
website/src/_includes/buttons-code-snippet.md | 32 +++
.../transforms/python/element-wise/filter.md | 186 ++------------
.../transforms/python/element-wise/flatmap.md | 267 +++------------------
8 files changed, 238 insertions(+), 491 deletions(-)
diff --git a/examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb b/examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb
index 7ffe297..f302d83 100644
--- a/examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb
+++ b/examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb
@@ -58,14 +58,12 @@
"localStorage.setItem('language', 'language-py')\n",
"</script>\n",
"\n",
- "</p><table align=\"left\" style=\"margin-right:1em\">\n",
+ "<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.Filter\">\n",
- " <img src=\"https://beam.apache.org/images/logos/sdks/python.png\" width=\"32px\" height=\"32px\" alt=\"Pydoc\"/>\n",
- " Pydoc\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.Filter\"><img src=\"https://beam.apache.org/images/logos/sdks/python.png\" width=\"32px\" height=\"32px\" alt=\"Pydoc\"/> Pydoc</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>\n",
"\n",
"Given a predicate, filter out all elements that don't satisfy that predicate.\n",
@@ -163,12 +161,10 @@
"source": [
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py\">\n",
- " <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/>\n",
- " View source code\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/> View source code</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>"
]
},
@@ -217,12 +213,10 @@
"source": [
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py\">\n",
- " <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/>\n",
- " View source code\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/> View source code</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>"
]
},
@@ -276,12 +270,10 @@
"source": [
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py\">\n",
- " <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/>\n",
- " View source code\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/> View source code</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>"
]
},
@@ -338,12 +330,10 @@
"source": [
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py\">\n",
- " <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/>\n",
- " View source code\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/> View source code</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>"
]
},
@@ -402,12 +392,10 @@
"source": [
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py\">\n",
- " <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/>\n",
- " View source code\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/> View source code</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>\n",
"\n",
"> **Note**: You can pass the `PCollection` as a *list* with `beam.pvalue.AsList(pcollection)`,\n",
@@ -470,12 +458,10 @@
"source": [
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py\">\n",
- " <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/>\n",
- " View source code\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/> View source code</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>"
]
},
@@ -494,12 +480,10 @@
"\n",
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.Filter\">\n",
- " <img src=\"https://beam.apache.org/images/logos/sdks/python.png\" width=\"32px\" height=\"32px\" alt=\"Pydoc\"/>\n",
- " Pydoc\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.Filter\"><img src=\"https://beam.apache.org/images/logos/sdks/python.png\" width=\"32px\" height=\"32px\" alt=\"Pydoc\"/> Pydoc</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>"
]
},
diff --git a/examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb b/examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb
index 8d5433b..c6e1fb2 100644
--- a/examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb
+++ b/examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb
@@ -58,14 +58,12 @@
"localStorage.setItem('language', 'language-py')\n",
"</script>\n",
"\n",
- "</p><table align=\"left\" style=\"margin-right:1em\">\n",
+ "<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.FlatMap\">\n",
- " <img src=\"https://beam.apache.org/images/logos/sdks/python.png\" width=\"32px\" height=\"32px\" alt=\"Pydoc\"/>\n",
- " Pydoc\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.FlatMap\"><img src=\"https://beam.apache.org/images/logos/sdks/python.png\" width=\"32px\" height=\"32px\" alt=\"Pydoc\"/> Pydoc</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>\n",
"\n",
"Applies a simple 1-to-many mapping function over each element in the collection.\n",
@@ -158,12 +156,10 @@
"source": [
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\">\n",
- " <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/>\n",
- " View source code\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/> View source code</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>"
]
},
@@ -211,12 +207,10 @@
"source": [
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\">\n",
- " <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/>\n",
- " View source code\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/> View source code</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>"
]
},
@@ -263,12 +257,10 @@
"source": [
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\">\n",
- " <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/>\n",
- " View source code\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/> View source code</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>"
]
},
@@ -319,12 +311,10 @@
"source": [
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\">\n",
- " <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/>\n",
- " View source code\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/> View source code</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>"
]
},
@@ -378,12 +368,10 @@
"source": [
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\">\n",
- " <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/>\n",
- " View source code\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/> View source code</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>"
]
},
@@ -434,12 +422,10 @@
"source": [
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\">\n",
- " <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/>\n",
- " View source code\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/> View source code</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>"
]
},
@@ -493,12 +479,10 @@
"source": [
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\">\n",
- " <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/>\n",
- " View source code\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/> View source code</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>"
]
},
@@ -562,12 +546,10 @@
"source": [
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\">\n",
- " <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/>\n",
- " View source code\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/> View source code</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>\n",
"\n",
"> **Note**: You can pass the `PCollection` as a *list* with `beam.pvalue.AsList(pcollection)`,\n",
@@ -635,12 +617,10 @@
"source": [
"<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\">\n",
- " <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/>\n",
- " View source code\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" width=\"32px\" height=\"32px\" alt=\"View source code\"/> View source code</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>"
]
},
@@ -658,14 +638,12 @@
" operation, and includes other abilities such as multiple output collections and side-inputs.\n",
"* [Map](https://beam.apache.org/documentation/transforms/python/elementwise/map) behaves the same, but produces exactly one output for each input.\n",
"\n",
- "<table>\n",
+ "<table align=\"left\" style=\"margin-right:1em\">\n",
" <td>\n",
- " <a class=\"button\" target=\"_blank\" href=\"https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.FlatMap\">\n",
- " <img src=\"https://beam.apache.org/images/logos/sdks/python.png\" width=\"32px\" height=\"32px\" alt=\"Pydoc\"/>\n",
- " Pydoc\n",
- " </a>\n",
+ " <a class=\"button\" target=\"_blank\" href=\"https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.FlatMap\"><img src=\"https://beam.apache.org/images/logos/sdks/python.png\" width=\"32px\" height=\"32px\" alt=\"Pydoc\"/> Pydoc</a>\n",
" </td>\n",
"</table>\n",
+ "\n",
"<br/><br/><br/>"
]
},
diff --git a/website/notebooks/generate.py b/website/notebooks/generate.py
index a01922b..76e69a7 100644
--- a/website/notebooks/generate.py
+++ b/website/notebooks/generate.py
@@ -29,16 +29,20 @@
# This creates the output notebooks in the `examples/notebooks` directory.
# You have to commit the generated notebooks after generating them.
-import argparse
+import logging
import md2ipynb
import nbformat
import os
+import sys
import yaml
docs_logo_url = 'https://beam.apache.org/images/logos/full-color/name-bottom/beam-logo-full-color-name-bottom-100.png'
-def run(docs, variables=None, inputs_dir='.', outputs_dir='.', imports_dir='.'):
+def run(docs, variables=None,
+ inputs_dir='.', outputs_dir='.', imports_dir='.', include_dir='.'):
+
+ errors = []
for basename, doc in docs.items():
languages=doc.get('languages', 'py java go').split()
for lang in languages:
@@ -54,27 +58,53 @@ def run(docs, variables=None, inputs_dir='.', outputs_dir='.', imports_dir='.'):
imports[0].insert(0, os.path.join(imports_dir, 'license.md'))
# Create a new notebook from the Markdown file contents.
+ input_file = basename + '.md'
ipynb_file = '/'.join([outputs_dir, '{}-{}.ipynb'.format(basename, lang)])
- notebook = md2ipynb.new_notebook(
- input_file=os.path.join(inputs_dir, basename + '.md'),
- variables=variables,
- imports=imports,
- notebook_title=doc.get('title', os.path.basename(basename).replace('-', ' ')),
- keep_classes=['language-' + lang, 'shell-sh'],
- docs_url='https://beam.apache.org/' + basename.replace('-', ''),
- docs_logo_url=docs_logo_url,
- github_ipynb_url='https://github.com/apache/beam/blob/master/' + ipynb_file,
- )
-
- # Write the notebook to file.
- output_dir = os.path.dirname(ipynb_file)
- if not os.path.exists(output_dir):
- os.makedirs(output_dir)
- with open(ipynb_file, 'w') as f:
- nbformat.write(notebook, f)
+ try:
+ notebook = md2ipynb.new_notebook(
+ input_file=os.path.join(inputs_dir, input_file),
+ variables=variables,
+ imports=imports,
+ notebook_title=doc.get('title', os.path.basename(basename).replace('-', ' ')),
+ keep_classes=['language-' + lang, 'shell-sh'],
+ docs_url='https://beam.apache.org/' + basename.replace('-', ''),
+ docs_logo_url=docs_logo_url,
+ github_ipynb_url='https://github.com/apache/beam/blob/master/' + ipynb_file,
+ include_dir=include_dir,
+ )
+ logging.info('{} succeeded'.format(input_file))
+
+ # Write the notebook to file.
+ output_dir = os.path.dirname(ipynb_file)
+ if not os.path.exists(output_dir):
+ os.makedirs(output_dir)
+ with open(ipynb_file, 'w') as f:
+ nbformat.write(notebook, f)
+ except Exception as e:
+ logging.error('{} failed: {}'.format(input_file, e))
+ errors.append((input_file, e))
+
+ if errors:
+ import traceback
+ sys.stdout.flush()
+ sys.stderr.flush()
+ print('')
+ print('=' * 60)
+ print(' Errors')
+ for input_file, e in errors:
+ print('')
+ print(input_file)
+ print('-' * len(input_file))
+ traceback.print_tb(e.__traceback__)
+
+ print('')
+ print('{} files processed ({} succeeded, {} failed)'.format(
+ len(docs), len(docs) - len(errors), len(errors)))
if __name__ == '__main__':
+ logging.basicConfig(level=logging.INFO)
+
script_dir = os.path.dirname(os.path.realpath(__file__))
root_dir = os.path.realpath(os.path.join(script_dir, '..', '..'))
@@ -87,7 +117,11 @@ if __name__ == '__main__':
variables = {'site': yaml.load(f.read())}
variables['site']['baseurl'] = variables['site']['url']
- inputs_dir = os.path.join(root_dir, 'website', 'src')
- outputs_dir = os.path.join(root_dir, 'examples', 'notebooks')
- imports_dir = os.path.join(script_dir, 'imports')
- run(docs, variables, inputs_dir, outputs_dir, imports_dir)
+ run(
+ docs=docs,
+ variables=variables,
+ inputs_dir=os.path.join(root_dir, 'website', 'src'),
+ outputs_dir=os.path.join(root_dir, 'examples', 'notebooks'),
+ imports_dir=os.path.join(script_dir, 'imports'),
+ include_dir=os.path.join(root_dir, 'website', 'src', '_includes'),
+ )
diff --git a/website/src/_includes/button-pydoc.md b/website/src/_includes/button-pydoc.md
new file mode 100644
index 0000000..c0135aa
--- /dev/null
+++ b/website/src/_includes/button-pydoc.md
@@ -0,0 +1,23 @@
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+{% capture button_url %}https://beam.apache.org/releases/pydoc/current/{{ include.path }}.html#{{ include.path }}.{{ include.class }}{% endcapture %}
+
+{% include button.md
+ url=button_url
+ logo="https://beam.apache.org/images/logos/sdks/python.png"
+ text="Pydoc"
+%}
+
+<br><br><br>
diff --git a/website/src/_includes/button.md b/website/src/_includes/button.md
new file mode 100644
index 0000000..0413771
--- /dev/null
+++ b/website/src/_includes/button.md
@@ -0,0 +1,21 @@
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+<div>
+<table align="left" style="margin-right:1em">
+ <td>
+ <a class="button" target="_blank" href="{{ include.url }}">{% if include.logo %}<img src="{{ include.logo }}" width="32px" height="32px" alt="{{ include.text }}" /> {% endif %}{{ include.text }}</a>
+ </td>
+</table>
+</div>
diff --git a/website/src/_includes/buttons-code-snippet.md b/website/src/_includes/buttons-code-snippet.md
new file mode 100644
index 0000000..055a730
--- /dev/null
+++ b/website/src/_includes/buttons-code-snippet.md
@@ -0,0 +1,32 @@
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+{% capture notebook_url %}https://colab.research.google.com/github/{{ site.branch_repo }}/{{ include.notebook }}{% endcapture %}
+
+{% capture code_url %}https://github.com/{{ site.branch_repo }}/{{ include.code }}{% endcapture %}
+
+{:.notebook-skip}
+{% include button.md
+ url=notebook_url
+ logo="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
+ text="Run code now"
+%}
+
+{% include button.md
+ url=code_url
+ logo="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
+ text="View source code"
+%}
+
+<br><br><br>
diff --git a/website/src/documentation/transforms/python/element-wise/filter.md b/website/src/documentation/transforms/python/element-wise/filter.md
index 384327a..346ffb4 100644
--- a/website/src/documentation/transforms/python/element-wise/filter.md
+++ b/website/src/documentation/transforms/python/element-wise/filter.md
@@ -24,17 +24,7 @@ limitations under the License.
localStorage.setItem('language', 'language-py')
</script>
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.Filter">
- <img src="https://beam.apache.org/images/logos/sdks/python.png"
- width="32px" height="32px" alt="Pydoc" />
- Pydoc
- </a>
- </td>
-</table>
-<br><br><br>
+{% include button-pydoc.md path="apache_beam.transforms.core" class="Filter" %}
Given a predicate, filter out all elements that don't satisfy that predicate.
May also be used to filter based on an inequality with a given value based
@@ -61,29 +51,10 @@ Output `PCollection` after `Filter`:
```
{% github_sample /apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter_test.py tag:perennials %}```
-{:.notebook-skip}
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://colab.research.google.com/github/{{ site.branch_repo }}/examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb">
- <img src="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
- width="32px" height="32px" alt="Run in Colab" />
- Run code now
- </a>
- </td>
-</table>
-
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://github.com/{{ site.branch_repo }}/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py">
- <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
- width="32px" height="32px" alt="View source code" />
- View source code
- </a>
- </td>
-</table>
-<br><br><br>
+{% include buttons-code-snippet.md
+ notebook="examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb"
+ code="sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py"
+%}
### Example 2: Filtering with a lambda function
@@ -99,29 +70,10 @@ Output `PCollection` after `Filter`:
```
{% github_sample /apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter_test.py tag:perennials %}```
-{:.notebook-skip}
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://colab.research.google.com/github/{{ site.branch_repo }}/examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb">
- <img src="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
- width="32px" height="32px" alt="Run code now" />
- Run code now
- </a>
- </td>
-</table>
-
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://github.com/{{ site.branch_repo }}/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py">
- <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
- width="32px" height="32px" alt="View source code" />
- View source code
- </a>
- </td>
-</table>
-<br><br><br>
+{% include buttons-code-snippet.md
+ notebook="examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb"
+ code="sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py"
+%}
### Example 3: Filtering with multiple arguments
@@ -140,29 +92,10 @@ Output `PCollection` after `Filter`:
```
{% github_sample /apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter_test.py tag:perennials %}```
-{:.notebook-skip}
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://colab.research.google.com/github/{{ site.branch_repo }}/examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb">
- <img src="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
- width="32px" height="32px" alt="Run in Colab" />
- Run code now
- </a>
- </td>
-</table>
-
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://github.com/{{ site.branch_repo }}/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py">
- <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
- width="32px" height="32px" alt="View source code" />
- View source code
- </a>
- </td>
-</table>
-<br><br><br>
+{% include buttons-code-snippet.md
+ notebook="examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb"
+ code="sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py"
+%}
### Example 4: Filtering with side inputs as singletons
@@ -182,29 +115,10 @@ Output `PCollection` after `Filter`:
```
{% github_sample /apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter_test.py tag:perennials %}```
-{:.notebook-skip}
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://colab.research.google.com/github/{{ site.branch_repo }}/examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb">
- <img src="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
- width="32px" height="32px" alt="Run in Colab" />
- Run code now
- </a>
- </td>
-</table>
-
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://github.com/{{ site.branch_repo }}/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py">
- <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
- width="32px" height="32px" alt="View source code" />
- View source code
- </a>
- </td>
-</table>
-<br><br><br>
+{% include buttons-code-snippet.md
+ notebook="examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb"
+ code="sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py"
+%}
### Example 5: Filtering with side inputs as iterators
@@ -222,29 +136,10 @@ Output `PCollection` after `Filter`:
```
{% github_sample /apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter_test.py tag:valid_plants %}```
-{:.notebook-skip}
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://colab.research.google.com/github/{{ site.branch_repo }}/examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb">
- <img src="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
- width="32px" height="32px" alt="Run in Colab" />
- Run code now
- </a>
- </td>
-</table>
-
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://github.com/{{ site.branch_repo }}/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py">
- <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
- width="32px" height="32px" alt="View source code" />
- View source code
- </a>
- </td>
-</table>
-<br><br><br>
+{% include buttons-code-snippet.md
+ notebook="examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb"
+ code="sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py"
+%}
> **Note**: You can pass the `PCollection` as a *list* with `beam.pvalue.AsList(pcollection)`,
> but this requires that all the elements fit into memory.
@@ -266,29 +161,10 @@ Output `PCollection` after `Filter`:
```
{% github_sample /apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter_test.py tag:perennials %}```
-{:.notebook-skip}
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://colab.research.google.com/github/{{ site.branch_repo }}/examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb">
- <img src="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
- width="32px" height="32px" alt="Run in Colab" />
- Run code now
- </a>
- </td>
-</table>
-
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://github.com/{{ site.branch_repo }}/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py">
- <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
- width="32px" height="32px" alt="View source code" />
- View source code
- </a>
- </td>
-</table>
-<br><br><br>
+{% include buttons-code-snippet.md
+ notebook="examples/notebooks/documentation/transforms/python/element-wise/filter-py.ipynb"
+ code="sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py"
+%}
## Related transforms
@@ -297,14 +173,4 @@ Output `PCollection` after `Filter`:
* [ParDo]({{ site.baseurl }}/documentation/transforms/python/elementwise/pardo) is the most general element-wise mapping
operation, and includes other abilities such as multiple output collections and side-inputs.
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.Filter">
- <img src="https://beam.apache.org/images/logos/sdks/python.png"
- width="32px" height="32px" alt="Pydoc" />
- Pydoc
- </a>
- </td>
-</table>
-<br><br><br>
+{% include button-pydoc.md path="apache_beam.transforms.core" class="Filter" %}
diff --git a/website/src/documentation/transforms/python/element-wise/flatmap.md b/website/src/documentation/transforms/python/element-wise/flatmap.md
index 58eb826..d2e861a 100644
--- a/website/src/documentation/transforms/python/element-wise/flatmap.md
+++ b/website/src/documentation/transforms/python/element-wise/flatmap.md
@@ -24,17 +24,7 @@ limitations under the License.
localStorage.setItem('language', 'language-py')
</script>
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.FlatMap">
- <img src="https://beam.apache.org/images/logos/sdks/python.png"
- width="32px" height="32px" alt="Pydoc" />
- Pydoc
- </a>
- </td>
-</table>
-<br><br><br>
+{% include button-pydoc.md path="apache_beam.transforms.core" class="FlatMap" %}
Applies a simple 1-to-many mapping function over each element in the collection.
The many elements are flattened into the resulting collection.
@@ -62,29 +52,10 @@ Output `PCollection` after `FlatMap`:
```
{% github_sample /apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map_test.py tag:plants %}```
-{:.notebook-skip}
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://colab.research.google.com/github/{{ site.branch_repo }}/examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb">
- <img src="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
- width="32px" height="32px" alt="Run code now" />
- Run code now
- </a>
- </td>
-</table>
-
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://github.com/{{ site.branch_repo }}/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py">
- <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
- width="32px" height="32px" alt="View source code" />
- View source code
- </a>
- </td>
-</table>
-<br><br><br>
+{% include buttons-code-snippet.md
+ notebook="examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb"
+ code="sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py"
+%}
### Example 2: FlatMap with a function
@@ -100,29 +71,10 @@ Output `PCollection` after `FlatMap`:
```
{% github_sample /apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map_test.py tag:plants %}```
-{:.notebook-skip}
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://colab.research.google.com/github/{{ site.branch_repo }}/examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb">
- <img src="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
- width="32px" height="32px" alt="Run code now" />
- Run code now
- </a>
- </td>
-</table>
-
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://github.com/{{ site.branch_repo }}/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py">
- <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
- width="32px" height="32px" alt="View source code" />
- View source code
- </a>
- </td>
-</table>
-<br><br><br>
+{% include buttons-code-snippet.md
+ notebook="examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb"
+ code="sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py"
+%}
### Example 3: FlatMap with a lambda function
@@ -140,29 +92,10 @@ Output `PCollection` after `FlatMap`:
```
{% github_sample /apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map_test.py tag:plants %}```
-{:.notebook-skip}
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://colab.research.google.com/github/{{ site.branch_repo }}/examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb">
- <img src="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
- width="32px" height="32px" alt="Run code now" />
- Run code now
- </a>
- </td>
-</table>
-
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://github.com/{{ site.branch_repo }}/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py">
- <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
- width="32px" height="32px" alt="View source code" />
- View source code
- </a>
- </td>
-</table>
-<br><br><br>
+{% include buttons-code-snippet.md
+ notebook="examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb"
+ code="sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py"
+%}
### Example 4: FlatMap with a generator
@@ -180,29 +113,10 @@ Output `PCollection` after `FlatMap`:
```
{% github_sample /apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map_test.py tag:plants %}```
-{:.notebook-skip}
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://colab.research.google.com/github/{{ site.branch_repo }}/examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb">
- <img src="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
- width="32px" height="32px" alt="Run code now" />
- Run code now
- </a>
- </td>
-</table>
-
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://github.com/{{ site.branch_repo }}/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py">
- <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
- width="32px" height="32px" alt="View source code" />
- View source code
- </a>
- </td>
-</table>
-<br><br><br>
+{% include buttons-code-snippet.md
+ notebook="examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb"
+ code="sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py"
+%}
### Example 5: FlatMapTuple for key-value pairs
@@ -219,29 +133,10 @@ Output `PCollection` after `FlatMapTuple`:
```
{% github_sample /apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map_test.py tag:plants %}```
-{:.notebook-skip}
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://colab.research.google.com/github/{{ site.branch_repo }}/examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb">
- <img src="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
- width="32px" height="32px" alt="Run code now" />
- Run code now
- </a>
- </td>
-</table>
-
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://github.com/{{ site.branch_repo }}/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py">
- <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
- width="32px" height="32px" alt="View source code" />
- View source code
- </a>
- </td>
-</table>
-<br><br><br>
+{% include buttons-code-snippet.md
+ notebook="examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb"
+ code="sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py"
+%}
### Example 6: FlatMap with multiple arguments
@@ -260,29 +155,10 @@ Output `PCollection` after `FlatMap`:
```
{% github_sample /apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map_test.py tag:plants %}```
-{:.notebook-skip}
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://colab.research.google.com/github/{{ site.branch_repo }}/examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb">
- <img src="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
- width="32px" height="32px" alt="Run code now" />
- Run code now
- </a>
- </td>
-</table>
-
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://github.com/{{ site.branch_repo }}/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py">
- <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
- width="32px" height="32px" alt="View source code" />
- View source code
- </a>
- </td>
-</table>
-<br><br><br>
+{% include buttons-code-snippet.md
+ notebook="examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb"
+ code="sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py"
+%}
### Example 7: FlatMap with side inputs as singletons
@@ -302,29 +178,10 @@ Output `PCollection` after `FlatMap`:
```
{% github_sample /apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map_test.py tag:plants %}```
-{:.notebook-skip}
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://colab.research.google.com/github/{{ site.branch_repo }}/examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb">
- <img src="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
- width="32px" height="32px" alt="Run code now" />
- Run code now
- </a>
- </td>
-</table>
-
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://github.com/{{ site.branch_repo }}/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py">
- <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
- width="32px" height="32px" alt="View source code" />
- View source code
- </a>
- </td>
-</table>
-<br><br><br>
+{% include buttons-code-snippet.md
+ notebook="examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb"
+ code="sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py"
+%}
### Example 8: FlatMap with side inputs as iterators
@@ -342,29 +199,10 @@ Output `PCollection` after `FlatMap`:
```
{% github_sample /apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map_test.py tag:valid_plants %}```
-{:.notebook-skip}
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://colab.research.google.com/github/{{ site.branch_repo }}/examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb">
- <img src="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
- width="32px" height="32px" alt="Run code now" />
- Run code now
- </a>
- </td>
-</table>
-
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://github.com/{{ site.branch_repo }}/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py">
- <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
- width="32px" height="32px" alt="View source code" />
- View source code
- </a>
- </td>
-</table>
-<br><br><br>
+{% include buttons-code-snippet.md
+ notebook="examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb"
+ code="sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py"
+%}
> **Note**: You can pass the `PCollection` as a *list* with `beam.pvalue.AsList(pcollection)`,
> but this requires that all the elements fit into memory.
@@ -386,29 +224,10 @@ Output `PCollection` after `FlatMap`:
```
{% github_sample /apache/beam/blob/master/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map_test.py tag:valid_plants %}```
-{:.notebook-skip}
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://colab.research.google.com/github/{{ site.branch_repo }}/examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb">
- <img src="https://github.com/googlecolab/open_in_colab/raw/master/images/icon32.png"
- width="32px" height="32px" alt="Run code now" />
- Run code now
- </a>
- </td>
-</table>
-
-<table align="left" style="margin-right:1em">
- <td>
- <a class="button" target="_blank"
- href="https://github.com/{{ site.branch_repo }}/sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py">
- <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png"
- width="32px" height="32px" alt="View source code" />
- View source code
- </a>
- </td>
-</table>
-<br><br><br>
+{% include buttons-code-snippet.md
+ notebook="examples/notebooks/documentation/transforms/python/element-wise/flatmap-py.ipynb"
+ code="sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py"
+%}
## Related transforms
@@ -418,14 +237,4 @@ Output `PCollection` after `FlatMap`:
operation, and includes other abilities such as multiple output collections and side-inputs.
* [Map]({{ site.baseurl }}/documentation/transforms/python/elementwise/map) behaves the same, but produces exactly one output for each input.
-<table>
- <td>
- <a class="button" target="_blank"
- href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.FlatMap">
- <img src="https://beam.apache.org/images/logos/sdks/python.png"
- width="32px" height="32px" alt="Pydoc" />
- Pydoc
- </a>
- </td>
-</table>
-<br><br><br>
+{% include button-pydoc.md path="apache_beam.transforms.core" class="FlatMap" %}