You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by al...@apache.org on 2017/05/24 23:18:44 UTC
[1/3] beam-site git commit: TikaIO JIRA number fix
Repository: beam-site
Updated Branches:
refs/heads/asf-site 0b8e34de4 -> f03fb601d
TikaIO JIRA number fix
Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/1f78f640
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/1f78f640
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/1f78f640
Branch: refs/heads/asf-site
Commit: 1f78f64026e26a714f0d4c22a48c676591782d6f
Parents: 0b8e34d
Author: sberyozkin <sb...@talend.com>
Authored: Wed May 24 21:27:21 2017 +0100
Committer: GitHub <no...@github.com>
Committed: Wed May 24 21:27:21 2017 +0100
----------------------------------------------------------------------
src/documentation/io/built-in.md | 4 ++++
1 file changed, 4 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/beam-site/blob/1f78f640/src/documentation/io/built-in.md
----------------------------------------------------------------------
diff --git a/src/documentation/io/built-in.md b/src/documentation/io/built-in.md
index 3bb5343..cdc57ae 100644
--- a/src/documentation/io/built-in.md
+++ b/src/documentation/io/built-in.md
@@ -124,4 +124,8 @@ This table contains I/O transforms that are currently planned or in-progress. St
<td>RestIO</td><td>Java</td>
<td><a href="https://issues.apache.org/jira/browse/BEAM-1946">BEAM-1946</a></td>
</tr>
+ <tr>
+ <td>TikaIO</td><td>Java</td>
+ <td><a href="https://issues.apache.org/jira/browse/BEAM-2328">BEAM-2328</a></td>
+ </tr>
</table>
[3/3] beam-site git commit: This closes #250
Posted by al...@apache.org.
This closes #250
Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/f03fb601
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/f03fb601
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/f03fb601
Branch: refs/heads/asf-site
Commit: f03fb601dda47c7ede157663ca140dfd1a5cc74d
Parents: 0b8e34d 047a210
Author: Ahmet Altay <al...@google.com>
Authored: Wed May 24 16:16:00 2017 -0700
Committer: Ahmet Altay <al...@google.com>
Committed: Wed May 24 16:16:00 2017 -0700
----------------------------------------------------------------------
content/documentation/io/built-in/index.html | 4 ++++
.../documentation/programming-guide/index.html | 22 ++++++++++----------
.../sdks/python-custom-io/index.html | 20 +++++++++---------
.../sdks/python-type-safety/index.html | 1 -
.../get-started/wordcount-example/index.html | 2 +-
src/documentation/io/built-in.md | 4 ++++
6 files changed, 30 insertions(+), 23 deletions(-)
----------------------------------------------------------------------
[2/3] beam-site git commit: Regenerate website
Posted by al...@apache.org.
Regenerate website
Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/047a2107
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/047a2107
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/047a2107
Branch: refs/heads/asf-site
Commit: 047a2107bdfacb5c2d89241c874ce033391e579a
Parents: 1f78f64
Author: Ahmet Altay <al...@google.com>
Authored: Wed May 24 16:16:00 2017 -0700
Committer: Ahmet Altay <al...@google.com>
Committed: Wed May 24 16:16:00 2017 -0700
----------------------------------------------------------------------
content/documentation/io/built-in/index.html | 4 ++++
.../documentation/programming-guide/index.html | 22 ++++++++++----------
.../sdks/python-custom-io/index.html | 20 +++++++++---------
.../sdks/python-type-safety/index.html | 1 -
.../get-started/wordcount-example/index.html | 2 +-
5 files changed, 26 insertions(+), 23 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/beam-site/blob/047a2107/content/documentation/io/built-in/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/io/built-in/index.html b/content/documentation/io/built-in/index.html
index 79e47f7..6b3de1b 100644
--- a/content/documentation/io/built-in/index.html
+++ b/content/documentation/io/built-in/index.html
@@ -255,6 +255,10 @@
<td>RestIO</td><td>Java</td>
<td><a href="https://issues.apache.org/jira/browse/BEAM-1946">BEAM-1946</a></td>
</tr>
+ <tr>
+ <td>TikaIO</td><td>Java</td>
+ <td><a href="https://issues.apache.org/jira/browse/BEAM-2328">BEAM-2328</a></td>
+ </tr>
</table>
</div>
http://git-wip-us.apache.org/repos/asf/beam-site/blob/047a2107/content/documentation/programming-guide/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/programming-guide/index.html b/content/documentation/programming-guide/index.html
index 56b5481..7b27b37 100644
--- a/content/documentation/programming-guide/index.html
+++ b/content/documentation/programming-guide/index.html
@@ -402,14 +402,14 @@
</code></pre>
</div>
-<div class="language-py highlighter-rouge"><pre class="highlight"><code><span class="n">p</span> <span class="o">=</span> <span class="n">beam</span><span class="o">.</span><span class="n">Pipeline</span><span class="p">(</span><span class="n">options</span><span class="o">=</span><span class="n">pipeline_options</span><span class="p">)</span>
+<div class="language-py highlighter-rouge"><pre class="highlight"><code><span class="k">with</span> <span class="n">beam</span><span class="o">.</span><span class="n">Pipeline</span><span class="p">(</span><span class="n">options</span><span class="o">=</span><span class="n">pipeline_options</span><span class="p">)</span> <span class="k">as</span> <span class="n">p</span><span class="p">:</span>
-<span class="n">lines</span> <span class="o">=</span> <span class="p">(</span><span class="n">p</span>
- <span class="o">|</span> <span class="n">beam</span><span class="o">.</span><span class="n">Create</span><span class="p">([</span>
- <span class="s">'To be, or not to be: that is the question: '</span><span class="p">,</span>
- <span class="s">'Whether </span><span class="se">\'</span><span class="s">tis nobler in the mind to suffer '</span><span class="p">,</span>
- <span class="s">'The slings and arrows of outrageous fortune, '</span><span class="p">,</span>
- <span class="s">'Or to take arms against a sea of troubles, '</span><span class="p">]))</span>
+ <span class="n">lines</span> <span class="o">=</span> <span class="p">(</span><span class="n">p</span>
+ <span class="o">|</span> <span class="n">beam</span><span class="o">.</span><span class="n">Create</span><span class="p">([</span>
+ <span class="s">'To be, or not to be: that is the question: '</span><span class="p">,</span>
+ <span class="s">'Whether </span><span class="se">\'</span><span class="s">tis nobler in the mind to suffer '</span><span class="p">,</span>
+ <span class="s">'The slings and arrows of outrageous fortune, '</span><span class="p">,</span>
+ <span class="s">'Or to take arms against a sea of troubles, '</span><span class="p">]))</span>
</code></pre>
</div>
@@ -1128,10 +1128,10 @@ guest, [[], [order4]]
<span class="n">lower_bound</span><span class="o">=</span><span class="n">pvalue</span><span class="o">.</span><span class="n">AsSingleton</span><span class="p">(</span><span class="n">avg_word_len</span><span class="p">)))</span>
<span class="c"># Mix and match.</span>
-<span class="n">small_but_nontrivial</span> <span class="o">=</span> <span class="n">words</span> <span class="o">|</span> <span class="n">beam</span><span class="o">.</span><span class="n">FlatMap</span><span class="p">(</span><span class="n">filter_using_length</span><span class="p">,</span>
- <span class="n">lower_bound</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span>
- <span class="n">upper_bound</span><span class="o">=</span><span class="n">pvalue</span><span class="o">.</span><span class="n">AsSingleton</span><span class="p">(</span>
- <span class="n">avg_word_len</span><span class="p">))</span>
+<span class="n">small_but_nontrivial</span> <span class="o">=</span> <span class="n">words</span> <span class="o">|</span> <span class="n">beam</span><span class="o">.</span><span class="n">FlatMap</span><span class="p">(</span>
+ <span class="n">filter_using_length</span><span class="p">,</span>
+ <span class="n">lower_bound</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span>
+ <span class="n">upper_bound</span><span class="o">=</span><span class="n">pvalue</span><span class="o">.</span><span class="n">AsSingleton</span><span class="p">(</span><span class="n">avg_word_len</span><span class="p">))</span>
<span class="c"># We can also pass side inputs to a ParDo transform, which will get passed to its process method.</span>
http://git-wip-us.apache.org/repos/asf/beam-site/blob/047a2107/content/documentation/sdks/python-custom-io/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/sdks/python-custom-io/index.html b/content/documentation/sdks/python-custom-io/index.html
index d6eb9f2..2952ea4 100644
--- a/content/documentation/sdks/python-custom-io/index.html
+++ b/content/documentation/sdks/python-custom-io/index.html
@@ -368,8 +368,8 @@
<p>To read data from the source in your pipeline, use the <code class="highlighter-rouge">Read</code> transform:</p>
-<div class="highlighter-rouge"><pre class="highlight"><code>p = beam.Pipeline(options=PipelineOptions())
-numbers = p | 'ProduceNumbers' >> beam.io.Read(CountingSource(count))
+<div class="highlighter-rouge"><pre class="highlight"><code>with beam.Pipeline(options=PipelineOptions()) as p:
+ numbers = p | 'ProduceNumbers' >> beam.io.Read(CountingSource(count))
</code></pre>
</div>
@@ -512,11 +512,11 @@ numbers = p | 'ProduceNumbers' >> beam.io.Read(CountingSource(count))
<p>The following code demonstrates how to write to the sink using the <code class="highlighter-rouge">Write</code> transform.</p>
-<div class="highlighter-rouge"><pre class="highlight"><code>p = beam.Pipeline(options=PipelineOptions())
-kvs = p | 'CreateKVs' >> beam.Create(KVs)
+<div class="highlighter-rouge"><pre class="highlight"><code>with beam.Pipeline(options=PipelineOptions()) as p:
+ kvs = p | 'CreateKVs' >> beam.Create(KVs)
-kvs | 'WriteToSimpleKV' >> beam.io.Write(
- SimpleKVSink('http://url_to_simple_kv/', final_table_name))
+ kvs | 'WriteToSimpleKV' >> beam.io.Write(
+ SimpleKVSink('http://url_to_simple_kv/', final_table_name))
</code></pre>
</div>
@@ -569,10 +569,10 @@ numbers = p | 'ProduceNumbers' >> ReadFromCountingSource(count)
<p>Finally, write to the sink:</p>
-<div class="highlighter-rouge"><pre class="highlight"><code>p = beam.Pipeline(options=PipelineOptions())
-kvs = p | 'CreateKVs' >> beam.core.Create(KVs)
-kvs | 'WriteToSimpleKV' >> WriteToKVSink(
- 'http://url_to_simple_kv/', final_table_name)
+<div class="highlighter-rouge"><pre class="highlight"><code>with beam.Pipeline(options=PipelineOptions()) as p:
+ kvs = p | 'CreateKVs' >> beam.core.Create(KVs)
+ kvs | 'WriteToSimpleKV' >> WriteToKVSink(
+ 'http://url_to_simple_kv/', final_table_name)
</code></pre>
</div>
http://git-wip-us.apache.org/repos/asf/beam-site/blob/047a2107/content/documentation/sdks/python-type-safety/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/sdks/python-type-safety/index.html b/content/documentation/sdks/python-type-safety/index.html
index 47395b2..53359ea 100644
--- a/content/documentation/sdks/python-type-safety/index.html
+++ b/content/documentation/sdks/python-type-safety/index.html
@@ -264,7 +264,6 @@ words_with_lens = words | MyTransform()
<p>For example, the following code would pass at both pipeline construction and runtime.</p>
<div class="highlighter-rouge"><pre class="highlight"><code>p | beam.Create(['a']) | beam.Map(lambda x: 3).with_output_types(str)
-p.run()
</code></pre>
</div>
http://git-wip-us.apache.org/repos/asf/beam-site/blob/047a2107/content/get-started/wordcount-example/index.html
----------------------------------------------------------------------
diff --git a/content/get-started/wordcount-example/index.html b/content/get-started/wordcount-example/index.html
index c73e22b..e4efeb7 100644
--- a/content/get-started/wordcount-example/index.html
+++ b/content/get-started/wordcount-example/index.html
@@ -491,7 +491,7 @@ Figure 1: The pipeline data flow.</p>
<span class="n">default</span><span class="o">=</span><span class="s">'gs://my-bucket/input'</span><span class="p">)</span>
<span class="n">options</span> <span class="o">=</span> <span class="n">PipelineOptions</span><span class="p">(</span><span class="n">argv</span><span class="p">)</span>
-<span class="n">p</span> <span class="o">=</span> <span class="n">beam</span><span class="o">.</span><span class="n">Pipeline</span><span class="p">(</span><span class="n">options</span><span class="o">=</span><span class="n">options</span><span class="p">)</span>
+<span class="k">with</span> <span class="n">beam</span><span class="o">.</span><span class="n">Pipeline</span><span class="p">(</span><span class="n">options</span><span class="o">=</span><span class="n">options</span><span class="p">)</span> <span class="k">as</span> <span class="n">p</span><span class="p">:</span>
</code></pre>
</div>