You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2020/10/14 06:04:10 UTC
[beam] branch asf-site updated: Publishing website 2020/10/14
06:03:42 at commit 7340bce
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new cf2feb2 Publishing website 2020/10/14 06:03:42 at commit 7340bce
cf2feb2 is described below
commit cf2feb21d9d0986a0938b30eb9c1230d4939d820
Author: jenkins <bu...@apache.org>
AuthorDate: Wed Oct 14 06:03:42 2020 +0000
Publishing website 2020/10/14 06:03:42 at commit 7340bce
---
.../dsls/sql/extensions/create-external-table/index.html | 13 ++++++++-----
website/generated-content/sitemap.xml | 2 +-
2 files changed, 9 insertions(+), 6 deletions(-)
diff --git a/website/generated-content/documentation/dsls/sql/extensions/create-external-table/index.html b/website/generated-content/documentation/dsls/sql/extensions/create-external-table/index.html
index abf53da..12e3900 100644
--- a/website/generated-content/documentation/dsls/sql/extensions/create-external-table/index.html
+++ b/website/generated-content/documentation/dsls/sql/extensions/create-external-table/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Beam SQL extension: CREATE EXTERNAL TABLE Statement</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integrati [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
<a href=/documentation/io/built-in/>external storage system</a>.
For some storage systems, <code>CREATE EXTERNAL TABLE</code> does not create a physical table until
a write occurs. After the physical table exists, you can access the table with
@@ -87,10 +87,13 @@ LOCATION 'projects/testing-integration/topics/user-location'
</code></pre><h2 id=kafka>Kafka</h2><p>KafkaIO is experimental in Beam SQL.</p><h3 id=syntax-3>Syntax</h3><pre><code>CREATE EXTERNAL TABLE [ IF NOT EXISTS ] tableName (tableElement [, tableElement ]*)
TYPE kafka
LOCATION 'kafka://localhost:2181/brokers'
-TBLPROPERTIES '{"bootstrap.servers":"localhost:9092", "topics": ["topic1", "topic2"]}'
+TBLPROPERTIES '{"bootstrap.servers":"localhost:9092", "topics": ["topic1", "topic2"], "format": "avro"}'
</code></pre><ul><li><code>LOCATION</code>: The Kafka topic URL.</li><li><code>TBLPROPERTIES</code>:<ul><li><code>bootstrap.servers</code>: Optional. Allows you to specify the bootstrap
-server.</li><li><code>topics</code>: Optional. Allows you to specify specific topics.</li></ul></li></ul><h3 id=read-mode-2>Read Mode</h3><p>Read Mode supports reading from a topic.</p><h3 id=write-mode-2>Write Mode</h3><p>Write Mode supports writing to a topic.</p><h3 id=supported-payload-1>Supported Payload</h3><ul><li>CSV<ul><li>Beam parses the messages, attempting to parse fields according to the
-types specified in the schema.</li></ul></li></ul><h3 id=schema-2>Schema</h3><p>Only simple types are supported.</p><h2 id=mongodb>MongoDB</h2><h3 id=syntax-4>Syntax</h3><pre><code>CREATE EXTERNAL TABLE [ IF NOT EXISTS ] tableName (tableElement [, tableElement ]*)
+server.</li><li><code>topics</code>: Optional. Allows you to specify specific topics.</li><li><code>format</code>: Optional. Allows you to specify the Kafka values format. Possible values are
+{<code>csv</code>, <code>avro</code>}. Defaults to <code>csv</code>.</li></ul></li></ul><h3 id=read-mode-2>Read Mode</h3><p>Read Mode supports reading from a topic.</p><h3 id=write-mode-2>Write Mode</h3><p>Write Mode supports writing to a topic.</p><h3 id=supported-payload-formats>Supported Payload Formats</h3><ul><li>CSV (default)<ul><li>Beam parses the messages, attempting to parse fields according to the
+types specified in the schema.</li></ul></li><li>Avro<ul><li>An Avro schema is automatically generated from the specified field
+types. It is used to parse incoming messages and to format outgoing
+messages.</li></ul></li></ul><h3 id=schema-2>Schema</h3><p>Only simple types are supported.</p><h2 id=mongodb>MongoDB</h2><h3 id=syntax-4>Syntax</h3><pre><code>CREATE EXTERNAL TABLE [ IF NOT EXISTS ] tableName (tableElement [, tableElement ]*)
TYPE mongodb
LOCATION 'mongodb://[HOST]:[PORT]/[DATABASE]/[COLLECTION]'
</code></pre><ul><li><code>LOCATION</code>: Location of the collection.<ul><li><code>HOST</code>: Location of the MongoDB server. Can be localhost or an ip address.
@@ -105,7 +108,7 @@ LOCATION '/home/admin/orders'
TBLPROPERTIES '{"format: "Excel"}'
</code></pre><ul><li><code>LOCATION</code>: The path to the file for Read Mode. The prefix for Write Mode.</li><li><code>TBLPROPERTIES</code>:<ul><li><code>format</code>: Optional. Allows you to specify the CSV Format, which controls
the field delimeter, quote character, record separator, and other properties.
-See the following table:</li></ul></li></ul><div class=table-bordered-wrapper><table><thead><tr><th>Value for <code>format</code></th><th>Field delimiter</th><th>Quote</th><th>Record separator</th><th>Ignore empty lines?</th><th>Allow missing column names?</th></tr></thead><tbody><tr><td><code>default</code></td><td><code>,</code></td><td><code>"</code></td><td><code>\r\n</code></td><td>Yes</td><td>No</td></tr><tr><td><code>rfc4180</code></td><td><code>,</code></td><td><code>"</code></td [...]
+See the following table:</li></ul></li></ul><div class=table-bordered-wrapper><table><thead><tr><th>Value for <code>format</code></th><th>Field delimiter</th><th>Quote</th><th>Record separator</th><th>Ignore empty lines?</th><th>Allow missing column names?</th></tr></thead><tbody><tr><td><code>default</code></td><td><code>,</code></td><td><code>"</code></td><td><code>\r\n</code></td><td>Yes</td><td>No</td></tr><tr><td><code>rfc4180</code></td><td><code>,</code></td><td><code>"</code></td [...]
types specified in the schema using org.apache.commons.csv.</li></ul></li></ul><h3 id=schema-4>Schema</h3><p>Only simple types are supported.</p><h3 id=example-3>Example</h3><pre><code>CREATE EXTERNAL TABLE orders (id INTEGER, price INTEGER)
TYPE text
LOCATION '/home/admin/orders'
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml
index dd62a32..6a19a7e 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.24.0/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/blog/p [...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.24.0/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/blog/p [...]
\ No newline at end of file