You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2020/01/29 17:24:44 UTC
[beam] branch asf-site updated: Publishing website 2020/01/29
17:24:36 at commit b97c45a
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 056443a Publishing website 2020/01/29 17:24:36 at commit b97c45a
056443a is described below
commit 056443a25c61e19aa18603b49c19dcc077bbfaaf
Author: jenkins <bu...@apache.org>
AuthorDate: Wed Jan 29 17:24:37 2020 +0000
Publishing website 2020/01/29 17:24:36 at commit b97c45a
---
.../io/built-in/google-bigquery/index.html | 26 +++++-----------------
1 file changed, 6 insertions(+), 20 deletions(-)
diff --git a/website/generated-content/documentation/io/built-in/google-bigquery/index.html b/website/generated-content/documentation/io/built-in/google-bigquery/index.html
index 6dbec7c..bf667c4 100644
--- a/website/generated-content/documentation/io/built-in/google-bigquery/index.html
+++ b/website/generated-content/documentation/io/built-in/google-bigquery/index.html
@@ -860,31 +860,22 @@ Beam’s support for the BigQuery Storage API has the following limitations:</p>
<ul>
<li>The SDK for Python does not support the BigQuery Storage API.</li>
- <li>Dynamic work re-balancing is not currently supported. As a result, reads might
-be less efficient in the presence of stragglers.</li>
<li>SDK versions 2.11.0 and 2.12.0 do not support reading with a query string; you
can only read from a table.</li>
+ <li>SDK versions before 2.15.0 do not support dynamic work rebalancing. As a
+result, reads might be less efficient in the presence of stragglers.</li>
</ul>
<p>Because this is currently a Beam experimental feature, export based reads are
recommended for production jobs.</p>
-<h4 id="enabling-the-api">Enabling the API</h4>
-
-<p>The BigQuery Storage API is distinct from the existing BigQuery API. You must
-<a href="https://cloud.google.com/bigquery/docs/reference/storage/#enabling_the_api">enable the BigQuery Storage API</a>
-for your Google Cloud Platform project.</p>
-
<h4 id="updating-your-code">Updating your code</h4>
<p>Use the following methods when you read from a table:</p>
<ul>
- <li>Required: Specify <a href="https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.TypedRead.html#withMethod-org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO.TypedRead.Method-">withMethod(Method.DIRECT_READ)</a> to use the BigQuery Storage API for
-the read operation.</li>
- <li>Optional: To use features such as <a href="https://cloud.google.com/bigquery/docs/reference/storage/">column projection and column filtering</a>,
-you must also specify a <a href="https://googleapis.github.io/google-cloud-java/google-api-grpc/apidocs/index.html?com/google/cloud/bigquery/storage/v1beta1/ReadOptions.TableReadOptions.html">TableReadOptions</a>
-proto using the <a href="https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.TypedRead.html#withReadOptions-com.google.cloud.bigquery.storage.v1beta1.ReadOptions.TableReadOptions-">withReadOptions</a> method.</li>
+ <li>Required: Specify <a href="https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.TypedRead.html#withMethod-org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO.TypedRead.Method-">withMethod(Method.DIRECT_READ)</a> to use the BigQuery Storage API for the read operation.</li>
+ <li>Optional: To use features such as <a href="https://cloud.google.com/bigquery/docs/reference/storage/">column projection and column filtering</a>, you must specify <a href="https://beam.apache.org/releases/javadoc/2.17.0/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.TypedRead.html#withSelectedFields-java.util.List-">withSelectedFields</a> and <a href="https://beam.apache.org/releases/javadoc/2.17.0/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.TypedRead.html#withRowRestriction-jav [...]
</ul>
<p>The following code snippet reads from a table. This example is from the <a href="https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/cookbook/BigQueryTornadoes.java">BigQueryTornadoes
@@ -894,17 +885,12 @@ the BigQuery Storage API and column projection to read public samples of weather
data from a BigQuery table. You can view the <a href="https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/cookbook/BigQueryTornadoes.java">full source code on
GitHub</a>.</p>
-<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">TableReadOptions</span> <span class="n">tableReadOptions</span> <span class="o">=</span>
- <span class="n">TableReadOptions</span><span class="o">.</span><span class="na">newBuilder</span><span class="o">()</span>
- <span class="o">.</span><span class="na">addAllSelectedFields</span><span class="o">(</span><span class="n">Lists</span><span class="o">.</span><span class="na">newArrayList</span><span class="o">(</span><span class="s">"month"</span><span class="o">,</span> <span class="s">"tornado"</span><span class="o">))</span>
- <span class="o">.</span><span class="na">build</span><span class="o">();</span>
-
- <span class="n">rowsFromBigQuery</span> <span class="o">=</span>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">rowsFromBigQuery</span> <span class="o">=</span>
<span class="n">p</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span>
<span class="n">BigQueryIO</span><span class="o">.</span><span class="na">readTableRows</span><span class="o">()</span>
<span class="o">.</span><span class="na">from</span><span class="o">(</span><span class="n">options</span><span class="o">.</span><span class="na">getInput</span><span class="o">())</span>
<span class="o">.</span><span class="na">withMethod</span><span class="o">(</span><span class="n">Method</span><span class="o">.</span><span class="na">DIRECT_READ</span><span class="o">)</span>
- <span class="o">.</span><span class="na">withReadOptions</span><span class="o">(</span><span class="n">tableReadOptions</span><span class="o">));</span>
+ <span class="o">.</span><span class="na">withSelectedFields</span><span class="o">(</span><span class="n">Lists</span><span class="o">.</span><span class="na">newArrayList</span><span class="o">(</span><span class="s">"month"</span><span class="o">,</span> <span class="s">"tornado"</span><span class="o">));</span>
</code></pre></div></div>
<div class="language-py highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># The SDK for Python does not support the BigQuery Storage API.</span>
</code></pre></div></div>