You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@carbondata.apache.org by ra...@apache.org on 2018/11/21 17:59:48 UTC
[21/50] [abbrv] carbondata git commit: [DOC] Update streaming-guide.md
[DOC] Update streaming-guide.md
Correct StreamSQL description
This closes #2910
Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/17de6001
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/17de6001
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/17de6001
Branch: refs/heads/branch-1.5
Commit: 17de6001c3f1151d1e4c0d3b7bcbd3e438db30f2
Parents: a160dfb
Author: Jacky Li <ja...@huawei.com>
Authored: Fri Nov 9 12:00:07 2018 +0800
Committer: ravipesala <ra...@gmail.com>
Committed: Wed Nov 21 22:40:11 2018 +0530
----------------------------------------------------------------------
docs/streaming-guide.md | 40 ++++++++++++++++++++++++++++++----------
1 file changed, 30 insertions(+), 10 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/carbondata/blob/17de6001/docs/streaming-guide.md
----------------------------------------------------------------------
diff --git a/docs/streaming-guide.md b/docs/streaming-guide.md
index 714b07a..0987ed2 100644
--- a/docs/streaming-guide.md
+++ b/docs/streaming-guide.md
@@ -31,9 +31,10 @@
- [StreamSQL](#streamsql)
- [Defining Streaming Table](#streaming-table)
- [Streaming Job Management](#streaming-job-management)
- - [START STREAM](#start-stream)
- - [STOP STREAM](#stop-stream)
+ - [CREATE STREAM](#create-stream)
+ - [DROP STREAM](#drop-stream)
- [SHOW STREAMS](#show-streams)
+ - [CLOSE STREAM](#close-stream)
## Quick example
Download and unzip spark-2.2.0-bin-hadoop2.7.tgz, and export $SPARK_HOME
@@ -333,7 +334,7 @@ Following example shows how to start a streaming ingest job
sql(
"""
- |START STREAM job123 ON TABLE sink
+ |CREATE STREAM job123 ON TABLE sink
|STMPROPERTIES(
| 'trigger'='ProcessingTime',
| 'interval'='1 seconds')
@@ -343,7 +344,7 @@ Following example shows how to start a streaming ingest job
| WHERE id % 2 = 1
""".stripMargin)
- sql("STOP STREAM job123")
+ sql("DROP STREAM job123")
sql("SHOW STREAMS [ON TABLE tableName]")
```
@@ -360,13 +361,13 @@ These two tables are normal carbon tables, they can be queried independently.
As above example shown:
-- `START STREAM jobName ON TABLE tableName` is used to start a streaming ingest job.
-- `STOP STREAM jobName` is used to stop a streaming job by its name
+- `CREATE STREAM jobName ON TABLE tableName` is used to start a streaming ingest job.
+- `DROP STREAM jobName` is used to stop a streaming job by its name
- `SHOW STREAMS [ON TABLE tableName]` is used to print streaming job information
-##### START STREAM
+##### CREATE STREAM
When this is issued, carbon will start a structured streaming job to do the streaming ingestion. Before launching the job, system will validate:
@@ -424,11 +425,25 @@ For Kafka data source, create the source table by:
)
```
+- Then CREATE STREAM can be used to start the streaming ingest job from source table to sink table
+```
+CREATE STREAM job123 ON TABLE sink
+STMPROPERTIES(
+ 'trigger'='ProcessingTime',
+ 'interval'='10 seconds'
+)
+AS
+ SELECT *
+ FROM source
+ WHERE id % 2 = 1
+```
-##### STOP STREAM
-
-When this is issued, the streaming job will be stopped immediately. It will fail if the jobName specified is not exist.
+##### DROP STREAM
+When `DROP STREAM` is issued, the streaming job will be stopped immediately. It will fail if the jobName specified is not exist.
+```
+DROP STREAM job123
+```
##### SHOW STREAMS
@@ -441,4 +456,9 @@ When this is issued, the streaming job will be stopped immediately. It will fail
`SHOW STREAMS` command will show all stream jobs in the system.
+##### ALTER TABLE CLOSE STREAM
+
+When the streaming application is stopped, and user want to manually trigger data conversion from carbon streaming files to columnar files, one can use
+`ALTER TABLE sink COMPACT 'CLOSE_STREAMING';`
+