You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2019/12/11 00:59:49 UTC
[beam] branch asf-site updated: Publishing website 2019/12/11
00:59:41 at commit 11c60b8
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new dcf3676 Publishing website 2019/12/11 00:59:41 at commit 11c60b8
dcf3676 is described below
commit dcf3676a00857826169f08fe153b223ffad65b0e
Author: jenkins <bu...@apache.org>
AuthorDate: Wed Dec 11 00:59:42 2019 +0000
Publishing website 2019/12/11 00:59:41 at commit 11c60b8
---
.../extensions/create-external-table/index.html | 31 +++++++++++++++++++---
1 file changed, 27 insertions(+), 4 deletions(-)
diff --git a/website/generated-content/documentation/dsls/sql/extensions/create-external-table/index.html b/website/generated-content/documentation/dsls/sql/extensions/create-external-table/index.html
index 2fc6503..c1a1eee 100644
--- a/website/generated-content/documentation/dsls/sql/extensions/create-external-table/index.html
+++ b/website/generated-content/documentation/dsls/sql/extensions/create-external-table/index.html
@@ -431,14 +431,26 @@ See the I/O specific sections for <code class="highlighter-rouge">tblProperties<
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CREATE EXTERNAL TABLE [ IF NOT EXISTS ] tableName (tableElement [, tableElement ]*)
TYPE bigquery
LOCATION '[PROJECT_ID]:[DATASET].[TABLE]'
+TBLPROPERTIES '{"method": "DEFAULT"}'
</code></pre></div></div>
<ul>
- <li><code class="highlighter-rouge">LOCATION:</code>Location of the table in the BigQuery CLI format.
+ <li><code class="highlighter-rouge">LOCATION</code>: Location of the table in the BigQuery CLI format.
<ul>
- <li><code class="highlighter-rouge">PROJECT_ID</code>: ID of the Google Cloud Project</li>
- <li><code class="highlighter-rouge">DATASET</code>: BigQuery Dataset ID</li>
- <li><code class="highlighter-rouge">TABLE</code>: BigQuery Table ID within the Dataset</li>
+ <li><code class="highlighter-rouge">PROJECT_ID</code>: ID of the Google Cloud Project.</li>
+ <li><code class="highlighter-rouge">DATASET</code>: BigQuery Dataset ID.</li>
+ <li><code class="highlighter-rouge">TABLE</code>: BigQuery Table ID within the Dataset.</li>
+ </ul>
+ </li>
+ <li><code class="highlighter-rouge">TBLPROPERTIES</code>:
+ <ul>
+ <li><code class="highlighter-rouge">method</code>: Optional. Read method to use. Following options are available:
+ <ul>
+ <li><code class="highlighter-rouge">DEFAULT</code>: If no property is set, will be used as default. Currently uses <code class="highlighter-rouge">EXPORT</code>.</li>
+ <li><code class="highlighter-rouge">DIRECT_READ</code>: Use the BigQuery Storage API.</li>
+ <li><code class="highlighter-rouge">EXPORT</code>: Export data to Google Cloud Storage in Avro format and read data files from that location.</li>
+ </ul>
+ </li>
</ul>
</li>
</ul>
@@ -448,6 +460,17 @@ LOCATION '[PROJECT_ID]:[DATASET].[TABLE]'
<p>Beam SQL supports reading columns with simple types (<code class="highlighter-rouge">simpleType</code>) and arrays of simple
types (<code class="highlighter-rouge">ARRAY<simpleType></code>).</p>
+<p>When reading using <code class="highlighter-rouge">EXPORT</code> method the following pipeline options should be set:</p>
+<ul>
+ <li><code class="highlighter-rouge">project</code>: ID of the Google Cloud Project.</li>
+ <li><code class="highlighter-rouge">tempLocation</code>: Bucket to store intermediate data in. Ex: <code class="highlighter-rouge">gs://temp-storage/temp</code>.</li>
+</ul>
+
+<p>When reading using <code class="highlighter-rouge">DIRECT_READ</code> method, an optimizer will attempt to perform
+project and predicate push-down, potentially reducing the time requited to read the data from BigQuery.</p>
+
+<p>More information about the BigQuery Storage API can be found <a href="https://beam.apache.org/documentation/io/built-in/google-bigquery/#storage-api">here</a>.</p>
+
<h3 id="write-mode">Write Mode</h3>
<p>if the table does not exist, Beam creates the table specified in location when