You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@carbondata.apache.org by ra...@apache.org on 2018/10/09 15:50:10 UTC
[08/45] carbondata git commit: [CARBONDATA-2957][DOC] update doc for
supporting compressor in table property
[CARBONDATA-2957][DOC] update doc for supporting compressor in table property
add doc for supporting compressor in table property
This closes #2744
Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/3cd8b947
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/3cd8b947
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/3cd8b947
Branch: refs/heads/branch-1.5
Commit: 3cd8b947c9d671f5d409f3b6c589c65afc2545ee
Parents: 3f99e9b
Author: xuchuanyin <xu...@hust.edu.cn>
Authored: Fri Sep 21 17:06:48 2018 +0800
Committer: kunal642 <ku...@gmail.com>
Committed: Wed Sep 26 18:13:57 2018 +0530
----------------------------------------------------------------------
docs/ddl-of-carbondata.md | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/carbondata/blob/3cd8b947/docs/ddl-of-carbondata.md
----------------------------------------------------------------------
diff --git a/docs/ddl-of-carbondata.md b/docs/ddl-of-carbondata.md
index 5eeba86..7cda9cd 100644
--- a/docs/ddl-of-carbondata.md
+++ b/docs/ddl-of-carbondata.md
@@ -32,6 +32,7 @@ CarbonData DDL statements are documented here,which includes:
* [Caching Level](#caching-at-block-or-blocklet-level)
* [Hive/Parquet folder Structure](#support-flat-folder-same-as-hiveparquet)
* [Extra Long String columns](#string-longer-than-32000-characters)
+ * [Compression for Table](#compression-for-table)
* [CREATE TABLE AS SELECT](#create-table-as-select)
* [CREATE EXTERNAL TABLE](#create-external-table)
* [External Table on Transactional table location](#create-external-table-on-managed-table-data-location)
@@ -424,6 +425,31 @@ CarbonData DDL statements are documented here,which includes:
**NOTE:** The LONG_STRING_COLUMNS can only be string/char/varchar columns and cannot be dictionary_include/sort_columns/complex columns.
+ - ##### Compression for table
+
+ Data compression is also supported by CarbonData.
+ By default, Snappy is used to compress the data. CarbonData also support ZSTD compressor.
+ User can specify the compressor in the table property:
+
+ ```
+ TBLPROPERTIES('carbon.column.compressor'='snappy')
+ ```
+ or
+ ```
+ TBLPROPERTIES('carbon.column.compressor'='zstd')
+ ```
+ If the compressor is configured, all the data loading and compaction will use that compressor.
+ If the compressor is not configured, the data loading and compaction will use the compressor from current system property.
+ In this scenario, the compressor for each load may differ if the system property is changed each time. This is helpful if you want to change the compressor for a table.
+ The corresponding system property is configured in carbon.properties file as below:
+ ```
+ carbon.column.compressor=snappy
+ ```
+ or
+ ```
+ carbon.column.compressor=zstd
+ ```
+
## CREATE TABLE AS SELECT
This function allows user to create a Carbon table from any of the Parquet/Hive/Carbon table. This is beneficial when the user wants to create Carbon table from any other Parquet/Hive table and use the Carbon query engine to query and achieve better query results for cases where Carbon is faster than other file formats. Also this feature can be used for backing up the data.