You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@carbondata.apache.org by ra...@apache.org on 2019/04/17 07:18:37 UTC
[carbondata-site] branch asf-site updated: Added 1.5.3 version
information
This is an automated email from the ASF dual-hosted git repository.
raghunandan pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/carbondata-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 3c19e68 Added 1.5.3 version information
3c19e68 is described below
commit 3c19e6890f828d50ac3234ab93401f216c9faa8d
Author: Raghunandan S <ca...@gmail.com>
AuthorDate: Mon Apr 15 15:22:00 2019 +0530
Added 1.5.3 version information
---
content/CSDK-guide.html | 6 +--
content/alluxio-guide.html | 6 +--
content/bloomfilter-datamap-guide.html | 6 +--
content/carbon-as-spark-datasource-guide.html | 6 +--
content/configuration-parameters.html | 6 +--
content/datamap-developer-guide.html | 6 +--
content/datamap-management.html | 7 ++--
content/ddl-of-carbondata.html | 43 ++++++++++++++++++++--
content/dml-of-carbondata.html | 30 +++++++++------
content/documentation.html | 6 +--
content/faq.html | 18 +++++++--
content/file-structure-of-carbondata.html | 6 +--
content/hive-guide.html | 6 +--
.../how-to-contribute-to-apache-carbondata.html | 6 +--
content/index.html | 32 ++++++----------
content/introduction.html | 6 +--
content/language-manual.html | 6 +--
content/lucene-datamap-guide.html | 6 +--
content/performance-tuning.html | 6 +--
content/preaggregate-datamap-guide.html | 6 +--
content/presto-guide.html | 36 +++++++++++-------
content/quick-start-guide.html | 22 ++++++-----
content/release-guide.html | 6 +--
content/s3-guide.html | 6 +--
content/sdk-guide.html | 6 +--
content/security.html | 9 ++---
content/segment-management-on-carbondata.html | 6 +--
content/streaming-guide.html | 6 +--
content/supported-data-types-in-carbondata.html | 6 +--
content/timeseries-datamap-guide.html | 6 +--
content/usecases.html | 10 ++---
content/videogallery.html | 9 ++---
src/main/scala/html/header.html | 6 +--
src/main/webapp/CSDK-guide.html | 6 +--
src/main/webapp/alluxio-guide.html | 6 +--
src/main/webapp/bloomfilter-datamap-guide.html | 6 +--
.../webapp/carbon-as-spark-datasource-guide.html | 6 +--
src/main/webapp/configuration-parameters.html | 6 +--
src/main/webapp/datamap-developer-guide.html | 6 +--
src/main/webapp/datamap-management.html | 7 ++--
src/main/webapp/ddl-of-carbondata.html | 43 ++++++++++++++++++++--
src/main/webapp/dml-of-carbondata.html | 30 +++++++++------
src/main/webapp/documentation.html | 6 +--
src/main/webapp/faq.html | 18 +++++++--
src/main/webapp/file-structure-of-carbondata.html | 6 +--
src/main/webapp/hive-guide.html | 6 +--
.../how-to-contribute-to-apache-carbondata.html | 6 +--
src/main/webapp/index.html | 6 +--
src/main/webapp/introduction.html | 6 +--
src/main/webapp/language-manual.html | 6 +--
src/main/webapp/lucene-datamap-guide.html | 6 +--
src/main/webapp/performance-tuning.html | 6 +--
src/main/webapp/preaggregate-datamap-guide.html | 6 +--
src/main/webapp/presto-guide.html | 36 +++++++++++-------
src/main/webapp/quick-start-guide.html | 22 ++++++-----
src/main/webapp/release-guide.html | 6 +--
src/main/webapp/s3-guide.html | 6 +--
src/main/webapp/sdk-guide.html | 6 +--
src/main/webapp/security.html | 9 ++---
.../webapp/segment-management-on-carbondata.html | 6 +--
src/main/webapp/streaming-guide.html | 6 +--
.../webapp/supported-data-types-in-carbondata.html | 6 +--
src/main/webapp/timeseries-datamap-guide.html | 6 +--
src/main/webapp/usecases.html | 10 ++---
src/main/webapp/videogallery.html | 9 ++---
src/site/markdown/datamap-management.md | 1 +
src/site/markdown/ddl-of-carbondata.md | 43 ++++++++++++++++++++++
src/site/markdown/dml-of-carbondata.md | 24 +++++++-----
src/site/markdown/faq.md | 18 +++++++++
src/site/markdown/presto-guide.md | 33 +++++++++++------
src/site/markdown/quick-start-guide.md | 6 ++-
src/site/markdown/usecases.md | 4 +-
72 files changed, 500 insertions(+), 305 deletions(-)
diff --git a/content/CSDK-guide.html b/content/CSDK-guide.html
index e7809e8..a7f2b76 100644
--- a/content/CSDK-guide.html
+++ b/content/CSDK-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/alluxio-guide.html b/content/alluxio-guide.html
index 037f29d..b6bcdd7 100644
--- a/content/alluxio-guide.html
+++ b/content/alluxio-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/bloomfilter-datamap-guide.html b/content/bloomfilter-datamap-guide.html
index b9a073f..cc7ee14 100644
--- a/content/bloomfilter-datamap-guide.html
+++ b/content/bloomfilter-datamap-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/carbon-as-spark-datasource-guide.html b/content/carbon-as-spark-datasource-guide.html
index 43698e5..9c8224a 100644
--- a/content/carbon-as-spark-datasource-guide.html
+++ b/content/carbon-as-spark-datasource-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/configuration-parameters.html b/content/configuration-parameters.html
index 6c48b5e..ca4b9e9 100644
--- a/content/configuration-parameters.html
+++ b/content/configuration-parameters.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/datamap-developer-guide.html b/content/datamap-developer-guide.html
index b0cb182..e5ec135 100644
--- a/content/datamap-developer-guide.html
+++ b/content/datamap-developer-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/datamap-management.html b/content/datamap-management.html
index ac847f3..172dc00 100644
--- a/content/datamap-management.html
+++ b/content/datamap-management.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
@@ -360,6 +360,7 @@ Scan Table: default.datamap1_table
<li>DataMapName</li>
<li>DataMapProviderName like mv, preaggreagte, timeseries, etc</li>
<li>Associated Table</li>
+<li>DataMap Properties</li>
</ul>
<h3>
<a id="compaction-on-datamap" class="anchor" href="#compaction-on-datamap" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Compaction on DataMap</h3>
diff --git a/content/ddl-of-carbondata.html b/content/ddl-of-carbondata.html
index 1ef64f8..8dea391 100644
--- a/content/ddl-of-carbondata.html
+++ b/content/ddl-of-carbondata.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
@@ -303,6 +303,9 @@
<li>
<p><a href="#bucketing">BUCKETING</a></p>
</li>
+<li>
+<p><a href="#cache">CACHE</a></p>
+</li>
</ul>
<h2>
<a id="create-table" class="anchor" href="#create-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CREATE TABLE</h2>
@@ -1288,6 +1291,40 @@ TBLPROPERTIES('BUCKETNUMBER'='noOfBuckets',
STORED AS carbondata
TBLPROPERTIES ('BUCKETNUMBER'='4', 'BUCKETCOLUMNS'='productName')
</code></pre>
+<h2>
+<a id="cache" class="anchor" href="#cache" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CACHE</h2>
+<p>CarbonData internally uses LRU caching to improve the performance. The user can get information
+about current cache used status in memory through the following command:</p>
+<div class="highlight highlight-source-sql"><pre>SHOW METACACHE</pre></div>
+<p>This shows the overall memory consumed in the cache by categories - index files, dictionary and
+datamaps. This also shows the cache usage by all the tables and children tables in the current
+database.</p>
+<div class="highlight highlight-source-sql"><pre>SHOW METACACHE <span class="pl-k">ON</span> TABLE tableName</pre></div>
+<p>This shows detailed information on cache usage by the table <code>tableName</code> and its carbonindex files,
+its dictionary files, its datamaps and children tables.</p>
+<p>This command is not allowed on child tables.</p>
+<div class="highlight highlight-source-sql"><pre> DROP METACACHE <span class="pl-k">ON</span> TABLE tableName</pre></div>
+<p>This clears any entry in cache by the table <code>tableName</code>, its carbonindex files,
+its dictionary files, its datamaps and children tables.</p>
+<p>This command is not allowed on child tables.</p>
+<h3>
+<a id="important-points" class="anchor" href="#important-points" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Important points</h3>
+<ol>
+<li>
+<p>Cache information is updated only after the select query is executed.</p>
+</li>
+<li>
+<p>In case of alter table the already loaded cache is invalidated when any subsequent select query
+is fired.</p>
+</li>
+<li>
+<p>Dictionary is loaded in cache only when the dictionary columns are queried upon. If we don't do
+direct query on dictionary column, cache will not be loaded.
+If we do <code>SELECT * FROM t1</code>, and even though for this case dictionary is loaded, it is loaded in
+executor and not on driver, and the final result rows are returned back to driver, and thus will
+produce no trace on driver cache if we do <code>SHOW METACACHE</code> or <code>SHOW METACACHE ON TABLE t1</code>.</p>
+</li>
+</ol>
<script>
$(function() {
// Show selected style on nav item
diff --git a/content/dml-of-carbondata.html b/content/dml-of-carbondata.html
index e765ecb..eaf4476 100644
--- a/content/dml-of-carbondata.html
+++ b/content/dml-of-carbondata.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
@@ -235,10 +235,13 @@
<h3>
<a id="load-files-to-carbondata-table" class="anchor" href="#load-files-to-carbondata-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>LOAD FILES TO CARBONDATA TABLE</h3>
<p>This command is used to load csv files to carbondata, OPTIONS are not mandatory for data loading process.</p>
-<pre><code>LOAD DATA [LOCAL] INPATH 'folder_path'
+<pre><code>LOAD DATA INPATH 'folder_path'
INTO TABLE [db_name.]table_name
OPTIONS(property_name=property_value, ...)
</code></pre>
+<p><strong>NOTE</strong>:
+* Use 'file://' prefix to indicate local input files path, but it just supports local mode.
+* If run on cluster mode, please upload all input files to distributed file system, for example 'hdfs://' for hdfs.</p>
<p><strong>Supported Properties:</strong></p>
<table>
<thead>
@@ -501,7 +504,7 @@ Used in case of nested Complex Map type. (eg., 'a\003b\002b\003c\001aa\003bb\002
<li>If this option is set to some invalid value other than TRUE or FALSE then it uses the default value.</li>
</ul>
<p>Example:</p>
-<pre><code>LOAD DATA local inpath '/opt/rawdata/data.csv' INTO table carbontable
+<pre><code>LOAD DATA inpath '/opt/rawdata/data.csv' INTO table carbontable
options('DELIMITER'=',', 'QUOTECHAR'='"','COMMENTCHAR'='#',
'HEADER'='false',
'FILEHEADER'='empno,empname,designation,doj,workgroupcategory,
@@ -610,15 +613,17 @@ It comes with the functionality to aggregate the records of a table by performin
<h3>
<a id="load-data-using-static-partition" class="anchor" href="#load-data-using-static-partition" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Load Data Using Static Partition</h3>
<p>This command allows you to load data using static partition.</p>
-<pre><code>LOAD DATA [LOCAL] INPATH 'folder_path'
+<pre><code>LOAD DATA INPATH 'folder_path'
INTO TABLE [db_name.]table_name PARTITION (partition_spec)
-OPTIONS(property_name=property_value, ...)
-INSERT INTO INTO TABLE [db_name.]table_name PARTITION (partition_spec) <SELECT STATEMENT>
+OPTIONS(property_name=property_value, ...)
+
+INSERT INTO TABLE [db_name.]table_name PARTITION (partition_spec) <SELECT STATEMENT>
</code></pre>
<p>Example:</p>
-<pre><code>LOAD DATA LOCAL INPATH '${env:HOME}/staticinput.csv'
+<pre><code>LOAD DATA INPATH '${env:HOME}/staticinput.csv'
INTO TABLE locationTable
-PARTITION (country = 'US', state = 'CA')
+PARTITION (country = 'US', state = 'CA')
+
INSERT INTO TABLE locationTable
PARTITION (country = 'US', state = 'AL')
SELECT <columns list excluding partition columns> FROM another_user
@@ -627,8 +632,9 @@ SELECT <columns list excluding partition columns> FROM another_user
<a id="load-data-using-dynamic-partition" class="anchor" href="#load-data-using-dynamic-partition" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Load Data Using Dynamic Partition</h3>
<p>This command allows you to load data using dynamic partition. If partition spec is not specified, then the partition is considered as dynamic.</p>
<p>Example:</p>
-<pre><code>LOAD DATA LOCAL INPATH '${env:HOME}/staticinput.csv'
-INTO TABLE locationTable
+<pre><code>LOAD DATA INPATH '${env:HOME}/staticinput.csv'
+INTO TABLE locationTable
+
INSERT INTO TABLE locationTable
SELECT <columns list excluding partition columns> FROM another_user
</code></pre>
diff --git a/content/documentation.html b/content/documentation.html
index 09db88f..b502f10 100644
--- a/content/documentation.html
+++ b/content/documentation.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/faq.html b/content/faq.html
index 7536a0b..49b793c 100644
--- a/content/faq.html
+++ b/content/faq.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
@@ -248,6 +248,7 @@
<li><a href="#failed-to-insert-data-on-the-cluster">Failed to insert data on the cluster</a></li>
<li><a href="#failed-to-execute-concurrent-operations-on-table-by-multiple-workers">Failed to execute Concurrent Operations(Load,Insert,Update) on table by multiple workers</a></li>
<li><a href="#failed-to-create-a-table-with-a-single-numeric-column">Failed to create a table with a single numeric column</a></li>
+<li><a href="#failed-to-create-datamap-and-drop-datamap-is-also-not-working">Failed to create datamap and drop datamap is also not working</a></li>
</ul>
<h2></h2>
<h2>
@@ -582,6 +583,17 @@ For example, you can use scp to copy this file to all the nodes.</p>
<p>Behaviour not supported.</p>
<p><strong>Procedure</strong></p>
<p>A single column that can be considered as dimension is mandatory for table creation.</p>
+<h2>
+<a id="failed-to-create-datamap-and-drop-datamap-is-also-not-working" class="anchor" href="#failed-to-create-datamap-and-drop-datamap-is-also-not-working" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Failed to create datamap and drop datamap is also not working</h2>
+<p><strong>Symptom</strong></p>
+<p>Execution fails with the following exception :</p>
+<pre><code>HDFS Quota Exceeded
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>HDFS Quota is set, and it is not letting carbondata write or modify any files.</p>
+<p><strong>Procedure</strong></p>
+<p>Drop that particular datamap using Drop Table command using table name as
+parentTableName_datamapName so as to clear the stale folders.</p>
<script>
// Show selected style on nav item
$(function() { $('.b-nav__faq').addClass('selected'); });
diff --git a/content/file-structure-of-carbondata.html b/content/file-structure-of-carbondata.html
index 3201546..10518af 100644
--- a/content/file-structure-of-carbondata.html
+++ b/content/file-structure-of-carbondata.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/hive-guide.html b/content/hive-guide.html
index 780c766..1c11ed7 100644
--- a/content/hive-guide.html
+++ b/content/hive-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/how-to-contribute-to-apache-carbondata.html b/content/how-to-contribute-to-apache-carbondata.html
index a8e5059..92ef5e4 100644
--- a/content/how-to-contribute-to-apache-carbondata.html
+++ b/content/how-to-contribute-to-apache-carbondata.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/index.html b/content/index.html
index ba38242..e1f33cb 100644
--- a/content/index.html
+++ b/content/index.html
@@ -54,6 +54,9 @@
class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -65,12 +68,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
- target="_blank">Apache CarbonData 1.3.1</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
@@ -316,6 +313,13 @@
</h4>
<div class="linkblock">
<div class="block-row">
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a>
+ <span class="release-date">Apr 2019</span>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Apache+CarbonData+1.5.3+Release"
+ class="whatsnew" target="_blank">what's new</a>
+ </div>
+ <div class="block-row">
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a>
<span class="release-date">Feb 2019</span>
@@ -343,20 +347,6 @@
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Apache+CarbonData+1.4.1+Release"
class="whatsnew" target="_blank">what's new</a>
</div>
- <div class="block-row">
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a>
- <span class="release-date">May 2018</span>
- <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Apache+CarbonData+1.4.0+Release"
- class="whatsnew" target="_blank">what's new</a>
- </div>
- <div class="block-row">
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
- target="_blank">Apache CarbonData 1.3.1</a>
- <span class="release-date">Mar 2018</span>
- <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Apache+CarbonData+1.3.1+Release"
- class="whatsnew" target="_blank">what's new</a>
- </div>
</div>
<div class="nextR">
<h4 class="title">Release Notes
@@ -488,7 +478,7 @@
to do is:</p>
<ol class="orderlist">
<li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
target="_blank">Download</a>the latest release.
</li>
diff --git a/content/introduction.html b/content/introduction.html
index 53e741c..8af7902 100644
--- a/content/introduction.html
+++ b/content/introduction.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/language-manual.html b/content/language-manual.html
index 9ac8add..0b7bca3 100644
--- a/content/language-manual.html
+++ b/content/language-manual.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/lucene-datamap-guide.html b/content/lucene-datamap-guide.html
index f9675e5..4f3290c 100644
--- a/content/lucene-datamap-guide.html
+++ b/content/lucene-datamap-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/performance-tuning.html b/content/performance-tuning.html
index 0a74864..1fde679 100644
--- a/content/performance-tuning.html
+++ b/content/performance-tuning.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/preaggregate-datamap-guide.html b/content/preaggregate-datamap-guide.html
index 9d7a387..b8c816d 100644
--- a/content/preaggregate-datamap-guide.html
+++ b/content/preaggregate-datamap-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/presto-guide.html b/content/presto-guide.html
index 00c3f5f..b104bdb 100644
--- a/content/presto-guide.html
+++ b/content/presto-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
@@ -227,22 +227,30 @@
<a id="presto-multinode-cluster-setup-for-carbondata" class="anchor" href="#presto-multinode-cluster-setup-for-carbondata" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Presto Multinode Cluster Setup for Carbondata</h2>
<h3>
<a id="installing-presto" class="anchor" href="#installing-presto" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Installing Presto</h3>
+<p>To know about which version of presto is supported by this version of carbon, visit
+<a href="https://github.com/apache/carbondata/blob/master/integration/presto/pom.xml" target=_blank>https://github.com/apache/carbondata/blob/master/integration/presto/pom.xml</a>
+and look for <code><presto.version></code></p>
+<p><em>Example:</em>
+<code><presto.version>0.217</presto.version></code>
+This means current version of carbon supports presto 0.217 version.</p>
+<p><em>Note:</em>
+Currently carbondata supports only one version of presto, cannot handle multiple versions at same time. If user wish to use older version of presto, then need to use older version of carbon (other old branches, say branch-1.5 and check the supported presto version in it's pom.xml file in integration/presto/)</p>
<ol>
-<li>Download the 0.210 version of Presto using:</li>
+<li>Download that version of Presto (say 0.217) using below command:</li>
</ol>
-<pre><code>wget https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.210/presto-server-0.210.tar.gz
+<pre><code>wget https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.217/presto-server-0.217.tar.gz
</code></pre>
<ol start="2">
<li>
-<p>Extract Presto tar file: <code>tar zxvf presto-server-0.210.tar.gz</code>.</p>
+<p>Extract Presto tar file: <code>tar zxvf presto-server-0.217.tar.gz</code>.</p>
</li>
<li>
-<p>Download the Presto CLI for the coordinator and name it presto.</p>
+<p>Download the Presto CLI of the same presto server version (say 0.217) for the coordinator and name it presto.</p>
</li>
</ol>
-<pre><code> wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.210/presto-cli-0.210-executable.jar
+<pre><code> wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.217/presto-cli-0.217-executable.jar
- mv presto-cli-0.210-executable.jar presto
+ mv presto-cli-0.217-executable.jar presto
chmod +x presto
</code></pre>
@@ -250,7 +258,7 @@
<a id="create-configuration-files" class="anchor" href="#create-configuration-files" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Create Configuration Files</h3>
<ol>
<li>
-<p>Create <code>etc</code> folder in presto-server-0.210 directory.</p>
+<p>Create <code>etc</code> folder in presto-server-0.217 directory.</p>
</li>
<li>
<p>Create <code>config.properties</code>, <code>jvm.config</code>, <code>log.properties</code>, and <code>node.properties</code> files.</p>
@@ -333,10 +341,10 @@ discovery.uri=<coordinator_ip>:8086
</ol>
<h3>
<a id="start-presto-server-on-all-nodes" class="anchor" href="#start-presto-server-on-all-nodes" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Start Presto Server on all nodes</h3>
-<pre><code>./presto-server-0.210/bin/launcher start
+<pre><code>./presto-server-0.217/bin/launcher start
</code></pre>
<p>To run it as a background process.</p>
-<pre><code>./presto-server-0.210/bin/launcher run
+<pre><code>./presto-server-0.217/bin/launcher run
</code></pre>
<p>To run it in foreground.</p>
<h3>
@@ -353,7 +361,7 @@ discovery.uri=<coordinator_ip>:8086
<h3>
<a id="config-presto-server" class="anchor" href="#config-presto-server" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Config presto server</h3>
<ul>
-<li>Download presto server (0.210 is suggested and supported) : <a href="https://repo1.maven.org/maven2/com/facebook/presto/presto-server/" target=_blank rel="nofollow">https://repo1.maven.org/maven2/com/facebook/presto/presto-server/</a>
+<li>Download presto server (0.217 is suggested and supported) : <a href="https://repo1.maven.org/maven2/com/facebook/presto/presto-server/" target=_blank rel="nofollow">https://repo1.maven.org/maven2/com/facebook/presto/presto-server/</a>
</li>
<li>Finish presto configuration following <a href="https://prestodb.io/docs/current/installation/deployment.html" target=_blank rel="nofollow">https://prestodb.io/docs/current/installation/deployment.html</a>.
A configuration example:</li>
@@ -448,7 +456,7 @@ carbondata files.</p>
<a id="query-carbondata-in-cli-of-presto" class="anchor" href="#query-carbondata-in-cli-of-presto" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Query carbondata in CLI of presto</h3>
<ul>
<li>
-<p>Download presto cli client of version 0.210 : <a href="https://repo1.maven.org/maven2/com/facebook/presto/presto-cli" target=_blank rel="nofollow">https://repo1.maven.org/maven2/com/facebook/presto/presto-cli</a></p>
+<p>Download presto cli client of version 0.217 : <a href="https://repo1.maven.org/maven2/com/facebook/presto/presto-cli" target=_blank rel="nofollow">https://repo1.maven.org/maven2/com/facebook/presto/presto-cli</a></p>
</li>
<li>
<p>Start CLI:</p>
diff --git a/content/quick-start-guide.html b/content/quick-start-guide.html
index 3b22632..61cedd3 100644
--- a/content/quick-start-guide.html
+++ b/content/quick-start-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
@@ -255,18 +255,18 @@ EOF
<p><a href="#installing-and-configuring-carbondata-on-presto">Installing and Configuring CarbonData on Presto</a></p>
<h4>
<a id="hive" class="anchor" href="#hive" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Hive</h4>
-<p><a href="./hive-guide.html" target=_blank>Installing and Configuring CarbonData on Hive</a></p>
+<p><a href="https://github.com/apache/carbondata/blob/master/docs/hive-guide.html" target=_blank>Installing and Configuring CarbonData on Hive</a></p>
<h3>
<a id="integration-with-storage-engines" class="anchor" href="#integration-with-storage-engines" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Integration with Storage Engines</h3>
<h4>
<a id="hdfs" class="anchor" href="#hdfs" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>HDFS</h4>
-<p><a href="#installing-and-configuring-carbondata-on-standalone-spark-cluster">CarbonData supports read and write with HDFS</a></p>
+<p><a href="https://github.com/apache/carbondata/blob/master/docs/quick-start-guide.html#installing-and-configuring-carbondata-on-standalone-spark-cluster">CarbonData supports read and write with HDFS</a></p>
<h4>
<a id="s3" class="anchor" href="#s3" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>S3</h4>
-<p><a href="./s3-guide.html" target=_blank>CarbonData supports read and write with S3</a></p>
+<p><a href="https://github.com/apache/carbondata/blob/master/docs/s3-guide.html" target=_blank>CarbonData supports read and write with S3</a></p>
<h4>
<a id="alluxio" class="anchor" href="#alluxio" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Alluxio</h4>
-<p><a href="./alluxio-guide.html" target=_blank>CarbonData supports read and write with Alluxio</a></p>
+<p><a href="https://github.com/apache/carbondata/blob/master/docs/alluxio-guide.html" target=_blank>CarbonData supports read and write with Alluxio</a></p>
<h2>
<a id="installing-and-configuring-carbondata-to-run-locally-with-spark-shell" class="anchor" href="#installing-and-configuring-carbondata-to-run-locally-with-spark-shell" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Installing and Configuring CarbonData to run locally with Spark Shell</h2>
<p>Apache Spark Shell provides a simple way to learn the API, as well as a powerful tool to analyze data interactively. Please visit <a href="http://spark.apache.org/docs/latest/" target=_blank rel="nofollow">Apache Spark Documentation</a> for more details on Spark shell.</p>
@@ -521,7 +521,11 @@ mv carbondata.tar.gz carbonlib/
--executor-memory 2G \
--executor-cores 2
</code></pre>
-<p><strong>NOTE</strong>: Make sure you have permissions for CarbonData JARs and files through which driver and executor will start.</p>
+<p><strong>NOTE</strong>:</p>
+<ul>
+<li>Make sure you have permissions for CarbonData JARs and files through which driver and executor will start.</li>
+<li>If use Spark + Hive 1.1.X, it needs to add carbondata assembly jar and carbondata-hive jar into parameter 'spark.sql.hive.metastore.jars' in spark-default.conf file.</li>
+</ul>
<h2>
<a id="query-execution-using-carbondata-thrift-server" class="anchor" href="#query-execution-using-carbondata-thrift-server" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Query Execution Using CarbonData Thrift Server</h2>
<h3>
@@ -748,4 +752,4 @@ $(function() { $('.b-nav__quickstart').addClass('selected'); });
</section><!-- End systemblock part -->
<script src="js/custom.js"></script>
</body>
-</html>
+</html>
\ No newline at end of file
diff --git a/content/release-guide.html b/content/release-guide.html
index ad94ba0..bbb02b5 100644
--- a/content/release-guide.html
+++ b/content/release-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/s3-guide.html b/content/s3-guide.html
index bf6f06d..41fd0ae 100644
--- a/content/s3-guide.html
+++ b/content/s3-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/sdk-guide.html b/content/sdk-guide.html
index 32bd876..23dd148 100644
--- a/content/sdk-guide.html
+++ b/content/sdk-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/security.html b/content/security.html
index dccfddf..b9ba692 100644
--- a/content/security.html
+++ b/content/security.html
@@ -45,6 +45,9 @@
aria-expanded="false">Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -56,12 +59,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
- target="_blank">Apache CarbonData 1.3.1</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/segment-management-on-carbondata.html b/content/segment-management-on-carbondata.html
index 30bca2e..b80b22c 100644
--- a/content/segment-management-on-carbondata.html
+++ b/content/segment-management-on-carbondata.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/streaming-guide.html b/content/streaming-guide.html
index 4c45380..7f3b414 100644
--- a/content/streaming-guide.html
+++ b/content/streaming-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/supported-data-types-in-carbondata.html b/content/supported-data-types-in-carbondata.html
index 26c0ff8..c00f28d 100644
--- a/content/supported-data-types-in-carbondata.html
+++ b/content/supported-data-types-in-carbondata.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/timeseries-datamap-guide.html b/content/timeseries-datamap-guide.html
index 281fd3a..b3770ee 100644
--- a/content/timeseries-datamap-guide.html
+++ b/content/timeseries-datamap-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/content/usecases.html b/content/usecases.html
index dc88c17..fa2c0ea 100644
--- a/content/usecases.html
+++ b/content/usecases.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
@@ -493,7 +493,7 @@
</tr>
<tr>
<td>Query</td>
-<td>enable.unsafe.in.query.procressing</td>
+<td>enable.unsafe.in.query.processing</td>
<td>true</td>
<td>Data that needs to be scanned in huge which in turn generates more short lived Java objects. This cause pressure of GC.using unsafe and offheap will reduce the GC overhead</td>
</tr>
@@ -501,7 +501,7 @@
<td>Query</td>
<td>use.offheap.in.query.processing</td>
<td>true</td>
-<td>Data that needs to be scanned in huge which in turn generates more short lived Java objects. This cause pressure of GC.using unsafe and offheap will reduce the GC overhead.offheap can be accessed through java unsafe.hence enable.unsafe.in.query.procressing needs to be true</td>
+<td>Data that needs to be scanned in huge which in turn generates more short lived Java objects. This cause pressure of GC.using unsafe and offheap will reduce the GC overhead.offheap can be accessed through java unsafe.hence enable.unsafe.in.query.processing needs to be true</td>
</tr>
<tr>
<td>Query</td>
diff --git a/content/videogallery.html b/content/videogallery.html
index 359be97..614d8c6 100644
--- a/content/videogallery.html
+++ b/content/videogallery.html
@@ -49,6 +49,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -60,12 +63,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
- target="_blank">Apache CarbonData 1.3.1</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/scala/html/header.html b/src/main/scala/html/header.html
index 6c9c9c7..b8079c9 100644
--- a/src/main/scala/html/header.html
+++ b/src/main/scala/html/header.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/CSDK-guide.html b/src/main/webapp/CSDK-guide.html
index e7809e8..a7f2b76 100644
--- a/src/main/webapp/CSDK-guide.html
+++ b/src/main/webapp/CSDK-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/alluxio-guide.html b/src/main/webapp/alluxio-guide.html
index 037f29d..b6bcdd7 100644
--- a/src/main/webapp/alluxio-guide.html
+++ b/src/main/webapp/alluxio-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/bloomfilter-datamap-guide.html b/src/main/webapp/bloomfilter-datamap-guide.html
index b9a073f..cc7ee14 100644
--- a/src/main/webapp/bloomfilter-datamap-guide.html
+++ b/src/main/webapp/bloomfilter-datamap-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/carbon-as-spark-datasource-guide.html b/src/main/webapp/carbon-as-spark-datasource-guide.html
index 43698e5..9c8224a 100644
--- a/src/main/webapp/carbon-as-spark-datasource-guide.html
+++ b/src/main/webapp/carbon-as-spark-datasource-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/configuration-parameters.html b/src/main/webapp/configuration-parameters.html
index 6c48b5e..ca4b9e9 100644
--- a/src/main/webapp/configuration-parameters.html
+++ b/src/main/webapp/configuration-parameters.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/datamap-developer-guide.html b/src/main/webapp/datamap-developer-guide.html
index b0cb182..e5ec135 100644
--- a/src/main/webapp/datamap-developer-guide.html
+++ b/src/main/webapp/datamap-developer-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/datamap-management.html b/src/main/webapp/datamap-management.html
index ac847f3..172dc00 100644
--- a/src/main/webapp/datamap-management.html
+++ b/src/main/webapp/datamap-management.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
@@ -360,6 +360,7 @@ Scan Table: default.datamap1_table
<li>DataMapName</li>
<li>DataMapProviderName like mv, preaggreagte, timeseries, etc</li>
<li>Associated Table</li>
+<li>DataMap Properties</li>
</ul>
<h3>
<a id="compaction-on-datamap" class="anchor" href="#compaction-on-datamap" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Compaction on DataMap</h3>
diff --git a/src/main/webapp/ddl-of-carbondata.html b/src/main/webapp/ddl-of-carbondata.html
index 1ef64f8..8dea391 100644
--- a/src/main/webapp/ddl-of-carbondata.html
+++ b/src/main/webapp/ddl-of-carbondata.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
@@ -303,6 +303,9 @@
<li>
<p><a href="#bucketing">BUCKETING</a></p>
</li>
+<li>
+<p><a href="#cache">CACHE</a></p>
+</li>
</ul>
<h2>
<a id="create-table" class="anchor" href="#create-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CREATE TABLE</h2>
@@ -1288,6 +1291,40 @@ TBLPROPERTIES('BUCKETNUMBER'='noOfBuckets',
STORED AS carbondata
TBLPROPERTIES ('BUCKETNUMBER'='4', 'BUCKETCOLUMNS'='productName')
</code></pre>
+<h2>
+<a id="cache" class="anchor" href="#cache" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CACHE</h2>
+<p>CarbonData internally uses LRU caching to improve the performance. The user can get information
+about current cache used status in memory through the following command:</p>
+<div class="highlight highlight-source-sql"><pre>SHOW METACACHE</pre></div>
+<p>This shows the overall memory consumed in the cache by categories - index files, dictionary and
+datamaps. This also shows the cache usage by all the tables and children tables in the current
+database.</p>
+<div class="highlight highlight-source-sql"><pre>SHOW METACACHE <span class="pl-k">ON</span> TABLE tableName</pre></div>
+<p>This shows detailed information on cache usage by the table <code>tableName</code> and its carbonindex files,
+its dictionary files, its datamaps and children tables.</p>
+<p>This command is not allowed on child tables.</p>
+<div class="highlight highlight-source-sql"><pre> DROP METACACHE <span class="pl-k">ON</span> TABLE tableName</pre></div>
+<p>This clears any entry in cache by the table <code>tableName</code>, its carbonindex files,
+its dictionary files, its datamaps and children tables.</p>
+<p>This command is not allowed on child tables.</p>
+<h3>
+<a id="important-points" class="anchor" href="#important-points" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Important points</h3>
+<ol>
+<li>
+<p>Cache information is updated only after the select query is executed.</p>
+</li>
+<li>
+<p>In case of alter table the already loaded cache is invalidated when any subsequent select query
+is fired.</p>
+</li>
+<li>
+<p>Dictionary is loaded in cache only when the dictionary columns are queried upon. If we don't do
+direct query on dictionary column, cache will not be loaded.
+If we do <code>SELECT * FROM t1</code>, and even though for this case dictionary is loaded, it is loaded in
+executor and not on driver, and the final result rows are returned back to driver, and thus will
+produce no trace on driver cache if we do <code>SHOW METACACHE</code> or <code>SHOW METACACHE ON TABLE t1</code>.</p>
+</li>
+</ol>
<script>
$(function() {
// Show selected style on nav item
diff --git a/src/main/webapp/dml-of-carbondata.html b/src/main/webapp/dml-of-carbondata.html
index e765ecb..eaf4476 100644
--- a/src/main/webapp/dml-of-carbondata.html
+++ b/src/main/webapp/dml-of-carbondata.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
@@ -235,10 +235,13 @@
<h3>
<a id="load-files-to-carbondata-table" class="anchor" href="#load-files-to-carbondata-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>LOAD FILES TO CARBONDATA TABLE</h3>
<p>This command is used to load csv files to carbondata, OPTIONS are not mandatory for data loading process.</p>
-<pre><code>LOAD DATA [LOCAL] INPATH 'folder_path'
+<pre><code>LOAD DATA INPATH 'folder_path'
INTO TABLE [db_name.]table_name
OPTIONS(property_name=property_value, ...)
</code></pre>
+<p><strong>NOTE</strong>:
+* Use 'file://' prefix to indicate local input files path, but it just supports local mode.
+* If run on cluster mode, please upload all input files to distributed file system, for example 'hdfs://' for hdfs.</p>
<p><strong>Supported Properties:</strong></p>
<table>
<thead>
@@ -501,7 +504,7 @@ Used in case of nested Complex Map type. (eg., 'a\003b\002b\003c\001aa\003bb\002
<li>If this option is set to some invalid value other than TRUE or FALSE then it uses the default value.</li>
</ul>
<p>Example:</p>
-<pre><code>LOAD DATA local inpath '/opt/rawdata/data.csv' INTO table carbontable
+<pre><code>LOAD DATA inpath '/opt/rawdata/data.csv' INTO table carbontable
options('DELIMITER'=',', 'QUOTECHAR'='"','COMMENTCHAR'='#',
'HEADER'='false',
'FILEHEADER'='empno,empname,designation,doj,workgroupcategory,
@@ -610,15 +613,17 @@ It comes with the functionality to aggregate the records of a table by performin
<h3>
<a id="load-data-using-static-partition" class="anchor" href="#load-data-using-static-partition" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Load Data Using Static Partition</h3>
<p>This command allows you to load data using static partition.</p>
-<pre><code>LOAD DATA [LOCAL] INPATH 'folder_path'
+<pre><code>LOAD DATA INPATH 'folder_path'
INTO TABLE [db_name.]table_name PARTITION (partition_spec)
-OPTIONS(property_name=property_value, ...)
-INSERT INTO INTO TABLE [db_name.]table_name PARTITION (partition_spec) <SELECT STATEMENT>
+OPTIONS(property_name=property_value, ...)
+
+INSERT INTO TABLE [db_name.]table_name PARTITION (partition_spec) <SELECT STATEMENT>
</code></pre>
<p>Example:</p>
-<pre><code>LOAD DATA LOCAL INPATH '${env:HOME}/staticinput.csv'
+<pre><code>LOAD DATA INPATH '${env:HOME}/staticinput.csv'
INTO TABLE locationTable
-PARTITION (country = 'US', state = 'CA')
+PARTITION (country = 'US', state = 'CA')
+
INSERT INTO TABLE locationTable
PARTITION (country = 'US', state = 'AL')
SELECT <columns list excluding partition columns> FROM another_user
@@ -627,8 +632,9 @@ SELECT <columns list excluding partition columns> FROM another_user
<a id="load-data-using-dynamic-partition" class="anchor" href="#load-data-using-dynamic-partition" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Load Data Using Dynamic Partition</h3>
<p>This command allows you to load data using dynamic partition. If partition spec is not specified, then the partition is considered as dynamic.</p>
<p>Example:</p>
-<pre><code>LOAD DATA LOCAL INPATH '${env:HOME}/staticinput.csv'
-INTO TABLE locationTable
+<pre><code>LOAD DATA INPATH '${env:HOME}/staticinput.csv'
+INTO TABLE locationTable
+
INSERT INTO TABLE locationTable
SELECT <columns list excluding partition columns> FROM another_user
</code></pre>
diff --git a/src/main/webapp/documentation.html b/src/main/webapp/documentation.html
index 09db88f..b502f10 100644
--- a/src/main/webapp/documentation.html
+++ b/src/main/webapp/documentation.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/faq.html b/src/main/webapp/faq.html
index 7536a0b..49b793c 100644
--- a/src/main/webapp/faq.html
+++ b/src/main/webapp/faq.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
@@ -248,6 +248,7 @@
<li><a href="#failed-to-insert-data-on-the-cluster">Failed to insert data on the cluster</a></li>
<li><a href="#failed-to-execute-concurrent-operations-on-table-by-multiple-workers">Failed to execute Concurrent Operations(Load,Insert,Update) on table by multiple workers</a></li>
<li><a href="#failed-to-create-a-table-with-a-single-numeric-column">Failed to create a table with a single numeric column</a></li>
+<li><a href="#failed-to-create-datamap-and-drop-datamap-is-also-not-working">Failed to create datamap and drop datamap is also not working</a></li>
</ul>
<h2></h2>
<h2>
@@ -582,6 +583,17 @@ For example, you can use scp to copy this file to all the nodes.</p>
<p>Behaviour not supported.</p>
<p><strong>Procedure</strong></p>
<p>A single column that can be considered as dimension is mandatory for table creation.</p>
+<h2>
+<a id="failed-to-create-datamap-and-drop-datamap-is-also-not-working" class="anchor" href="#failed-to-create-datamap-and-drop-datamap-is-also-not-working" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Failed to create datamap and drop datamap is also not working</h2>
+<p><strong>Symptom</strong></p>
+<p>Execution fails with the following exception :</p>
+<pre><code>HDFS Quota Exceeded
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>HDFS Quota is set, and it is not letting carbondata write or modify any files.</p>
+<p><strong>Procedure</strong></p>
+<p>Drop that particular datamap using Drop Table command using table name as
+parentTableName_datamapName so as to clear the stale folders.</p>
<script>
// Show selected style on nav item
$(function() { $('.b-nav__faq').addClass('selected'); });
diff --git a/src/main/webapp/file-structure-of-carbondata.html b/src/main/webapp/file-structure-of-carbondata.html
index 3201546..10518af 100644
--- a/src/main/webapp/file-structure-of-carbondata.html
+++ b/src/main/webapp/file-structure-of-carbondata.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/hive-guide.html b/src/main/webapp/hive-guide.html
index 780c766..1c11ed7 100644
--- a/src/main/webapp/hive-guide.html
+++ b/src/main/webapp/hive-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/how-to-contribute-to-apache-carbondata.html b/src/main/webapp/how-to-contribute-to-apache-carbondata.html
index a8e5059..92ef5e4 100644
--- a/src/main/webapp/how-to-contribute-to-apache-carbondata.html
+++ b/src/main/webapp/how-to-contribute-to-apache-carbondata.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/index.html b/src/main/webapp/index.html
index 047370c..6f7554d 100644
--- a/src/main/webapp/index.html
+++ b/src/main/webapp/index.html
@@ -54,6 +54,9 @@
class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -65,9 +68,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/introduction.html b/src/main/webapp/introduction.html
index 53e741c..8af7902 100644
--- a/src/main/webapp/introduction.html
+++ b/src/main/webapp/introduction.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/language-manual.html b/src/main/webapp/language-manual.html
index 9ac8add..0b7bca3 100644
--- a/src/main/webapp/language-manual.html
+++ b/src/main/webapp/language-manual.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/lucene-datamap-guide.html b/src/main/webapp/lucene-datamap-guide.html
index f9675e5..4f3290c 100644
--- a/src/main/webapp/lucene-datamap-guide.html
+++ b/src/main/webapp/lucene-datamap-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/performance-tuning.html b/src/main/webapp/performance-tuning.html
index 0a74864..1fde679 100644
--- a/src/main/webapp/performance-tuning.html
+++ b/src/main/webapp/performance-tuning.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/preaggregate-datamap-guide.html b/src/main/webapp/preaggregate-datamap-guide.html
index 9d7a387..b8c816d 100644
--- a/src/main/webapp/preaggregate-datamap-guide.html
+++ b/src/main/webapp/preaggregate-datamap-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/presto-guide.html b/src/main/webapp/presto-guide.html
index 00c3f5f..b104bdb 100644
--- a/src/main/webapp/presto-guide.html
+++ b/src/main/webapp/presto-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
@@ -227,22 +227,30 @@
<a id="presto-multinode-cluster-setup-for-carbondata" class="anchor" href="#presto-multinode-cluster-setup-for-carbondata" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Presto Multinode Cluster Setup for Carbondata</h2>
<h3>
<a id="installing-presto" class="anchor" href="#installing-presto" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Installing Presto</h3>
+<p>To know about which version of presto is supported by this version of carbon, visit
+<a href="https://github.com/apache/carbondata/blob/master/integration/presto/pom.xml" target=_blank>https://github.com/apache/carbondata/blob/master/integration/presto/pom.xml</a>
+and look for <code><presto.version></code></p>
+<p><em>Example:</em>
+<code><presto.version>0.217</presto.version></code>
+This means current version of carbon supports presto 0.217 version.</p>
+<p><em>Note:</em>
+Currently carbondata supports only one version of presto, cannot handle multiple versions at same time. If user wish to use older version of presto, then need to use older version of carbon (other old branches, say branch-1.5 and check the supported presto version in it's pom.xml file in integration/presto/)</p>
<ol>
-<li>Download the 0.210 version of Presto using:</li>
+<li>Download that version of Presto (say 0.217) using below command:</li>
</ol>
-<pre><code>wget https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.210/presto-server-0.210.tar.gz
+<pre><code>wget https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.217/presto-server-0.217.tar.gz
</code></pre>
<ol start="2">
<li>
-<p>Extract Presto tar file: <code>tar zxvf presto-server-0.210.tar.gz</code>.</p>
+<p>Extract Presto tar file: <code>tar zxvf presto-server-0.217.tar.gz</code>.</p>
</li>
<li>
-<p>Download the Presto CLI for the coordinator and name it presto.</p>
+<p>Download the Presto CLI of the same presto server version (say 0.217) for the coordinator and name it presto.</p>
</li>
</ol>
-<pre><code> wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.210/presto-cli-0.210-executable.jar
+<pre><code> wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.217/presto-cli-0.217-executable.jar
- mv presto-cli-0.210-executable.jar presto
+ mv presto-cli-0.217-executable.jar presto
chmod +x presto
</code></pre>
@@ -250,7 +258,7 @@
<a id="create-configuration-files" class="anchor" href="#create-configuration-files" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Create Configuration Files</h3>
<ol>
<li>
-<p>Create <code>etc</code> folder in presto-server-0.210 directory.</p>
+<p>Create <code>etc</code> folder in presto-server-0.217 directory.</p>
</li>
<li>
<p>Create <code>config.properties</code>, <code>jvm.config</code>, <code>log.properties</code>, and <code>node.properties</code> files.</p>
@@ -333,10 +341,10 @@ discovery.uri=<coordinator_ip>:8086
</ol>
<h3>
<a id="start-presto-server-on-all-nodes" class="anchor" href="#start-presto-server-on-all-nodes" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Start Presto Server on all nodes</h3>
-<pre><code>./presto-server-0.210/bin/launcher start
+<pre><code>./presto-server-0.217/bin/launcher start
</code></pre>
<p>To run it as a background process.</p>
-<pre><code>./presto-server-0.210/bin/launcher run
+<pre><code>./presto-server-0.217/bin/launcher run
</code></pre>
<p>To run it in foreground.</p>
<h3>
@@ -353,7 +361,7 @@ discovery.uri=<coordinator_ip>:8086
<h3>
<a id="config-presto-server" class="anchor" href="#config-presto-server" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Config presto server</h3>
<ul>
-<li>Download presto server (0.210 is suggested and supported) : <a href="https://repo1.maven.org/maven2/com/facebook/presto/presto-server/" target=_blank rel="nofollow">https://repo1.maven.org/maven2/com/facebook/presto/presto-server/</a>
+<li>Download presto server (0.217 is suggested and supported) : <a href="https://repo1.maven.org/maven2/com/facebook/presto/presto-server/" target=_blank rel="nofollow">https://repo1.maven.org/maven2/com/facebook/presto/presto-server/</a>
</li>
<li>Finish presto configuration following <a href="https://prestodb.io/docs/current/installation/deployment.html" target=_blank rel="nofollow">https://prestodb.io/docs/current/installation/deployment.html</a>.
A configuration example:</li>
@@ -448,7 +456,7 @@ carbondata files.</p>
<a id="query-carbondata-in-cli-of-presto" class="anchor" href="#query-carbondata-in-cli-of-presto" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Query carbondata in CLI of presto</h3>
<ul>
<li>
-<p>Download presto cli client of version 0.210 : <a href="https://repo1.maven.org/maven2/com/facebook/presto/presto-cli" target=_blank rel="nofollow">https://repo1.maven.org/maven2/com/facebook/presto/presto-cli</a></p>
+<p>Download presto cli client of version 0.217 : <a href="https://repo1.maven.org/maven2/com/facebook/presto/presto-cli" target=_blank rel="nofollow">https://repo1.maven.org/maven2/com/facebook/presto/presto-cli</a></p>
</li>
<li>
<p>Start CLI:</p>
diff --git a/src/main/webapp/quick-start-guide.html b/src/main/webapp/quick-start-guide.html
index 3b22632..61cedd3 100644
--- a/src/main/webapp/quick-start-guide.html
+++ b/src/main/webapp/quick-start-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
@@ -255,18 +255,18 @@ EOF
<p><a href="#installing-and-configuring-carbondata-on-presto">Installing and Configuring CarbonData on Presto</a></p>
<h4>
<a id="hive" class="anchor" href="#hive" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Hive</h4>
-<p><a href="./hive-guide.html" target=_blank>Installing and Configuring CarbonData on Hive</a></p>
+<p><a href="https://github.com/apache/carbondata/blob/master/docs/hive-guide.html" target=_blank>Installing and Configuring CarbonData on Hive</a></p>
<h3>
<a id="integration-with-storage-engines" class="anchor" href="#integration-with-storage-engines" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Integration with Storage Engines</h3>
<h4>
<a id="hdfs" class="anchor" href="#hdfs" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>HDFS</h4>
-<p><a href="#installing-and-configuring-carbondata-on-standalone-spark-cluster">CarbonData supports read and write with HDFS</a></p>
+<p><a href="https://github.com/apache/carbondata/blob/master/docs/quick-start-guide.html#installing-and-configuring-carbondata-on-standalone-spark-cluster">CarbonData supports read and write with HDFS</a></p>
<h4>
<a id="s3" class="anchor" href="#s3" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>S3</h4>
-<p><a href="./s3-guide.html" target=_blank>CarbonData supports read and write with S3</a></p>
+<p><a href="https://github.com/apache/carbondata/blob/master/docs/s3-guide.html" target=_blank>CarbonData supports read and write with S3</a></p>
<h4>
<a id="alluxio" class="anchor" href="#alluxio" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Alluxio</h4>
-<p><a href="./alluxio-guide.html" target=_blank>CarbonData supports read and write with Alluxio</a></p>
+<p><a href="https://github.com/apache/carbondata/blob/master/docs/alluxio-guide.html" target=_blank>CarbonData supports read and write with Alluxio</a></p>
<h2>
<a id="installing-and-configuring-carbondata-to-run-locally-with-spark-shell" class="anchor" href="#installing-and-configuring-carbondata-to-run-locally-with-spark-shell" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Installing and Configuring CarbonData to run locally with Spark Shell</h2>
<p>Apache Spark Shell provides a simple way to learn the API, as well as a powerful tool to analyze data interactively. Please visit <a href="http://spark.apache.org/docs/latest/" target=_blank rel="nofollow">Apache Spark Documentation</a> for more details on Spark shell.</p>
@@ -521,7 +521,11 @@ mv carbondata.tar.gz carbonlib/
--executor-memory 2G \
--executor-cores 2
</code></pre>
-<p><strong>NOTE</strong>: Make sure you have permissions for CarbonData JARs and files through which driver and executor will start.</p>
+<p><strong>NOTE</strong>:</p>
+<ul>
+<li>Make sure you have permissions for CarbonData JARs and files through which driver and executor will start.</li>
+<li>If use Spark + Hive 1.1.X, it needs to add carbondata assembly jar and carbondata-hive jar into parameter 'spark.sql.hive.metastore.jars' in spark-default.conf file.</li>
+</ul>
<h2>
<a id="query-execution-using-carbondata-thrift-server" class="anchor" href="#query-execution-using-carbondata-thrift-server" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Query Execution Using CarbonData Thrift Server</h2>
<h3>
@@ -748,4 +752,4 @@ $(function() { $('.b-nav__quickstart').addClass('selected'); });
</section><!-- End systemblock part -->
<script src="js/custom.js"></script>
</body>
-</html>
+</html>
\ No newline at end of file
diff --git a/src/main/webapp/release-guide.html b/src/main/webapp/release-guide.html
index ad94ba0..bbb02b5 100644
--- a/src/main/webapp/release-guide.html
+++ b/src/main/webapp/release-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/s3-guide.html b/src/main/webapp/s3-guide.html
index bf6f06d..41fd0ae 100644
--- a/src/main/webapp/s3-guide.html
+++ b/src/main/webapp/s3-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/sdk-guide.html b/src/main/webapp/sdk-guide.html
index 32bd876..23dd148 100644
--- a/src/main/webapp/sdk-guide.html
+++ b/src/main/webapp/sdk-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/security.html b/src/main/webapp/security.html
index dccfddf..b9ba692 100644
--- a/src/main/webapp/security.html
+++ b/src/main/webapp/security.html
@@ -45,6 +45,9 @@
aria-expanded="false">Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -56,12 +59,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
- target="_blank">Apache CarbonData 1.3.1</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/segment-management-on-carbondata.html b/src/main/webapp/segment-management-on-carbondata.html
index 30bca2e..b80b22c 100644
--- a/src/main/webapp/segment-management-on-carbondata.html
+++ b/src/main/webapp/segment-management-on-carbondata.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/streaming-guide.html b/src/main/webapp/streaming-guide.html
index 4c45380..7f3b414 100644
--- a/src/main/webapp/streaming-guide.html
+++ b/src/main/webapp/streaming-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/supported-data-types-in-carbondata.html b/src/main/webapp/supported-data-types-in-carbondata.html
index 26c0ff8..c00f28d 100644
--- a/src/main/webapp/supported-data-types-in-carbondata.html
+++ b/src/main/webapp/supported-data-types-in-carbondata.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/timeseries-datamap-guide.html b/src/main/webapp/timeseries-datamap-guide.html
index 281fd3a..b3770ee 100644
--- a/src/main/webapp/timeseries-datamap-guide.html
+++ b/src/main/webapp/timeseries-datamap-guide.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/main/webapp/usecases.html b/src/main/webapp/usecases.html
index dc88c17..fa2c0ea 100644
--- a/src/main/webapp/usecases.html
+++ b/src/main/webapp/usecases.html
@@ -52,6 +52,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -63,9 +66,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
@@ -493,7 +493,7 @@
</tr>
<tr>
<td>Query</td>
-<td>enable.unsafe.in.query.procressing</td>
+<td>enable.unsafe.in.query.processing</td>
<td>true</td>
<td>Data that needs to be scanned in huge which in turn generates more short lived Java objects. This cause pressure of GC.using unsafe and offheap will reduce the GC overhead</td>
</tr>
@@ -501,7 +501,7 @@
<td>Query</td>
<td>use.offheap.in.query.processing</td>
<td>true</td>
-<td>Data that needs to be scanned in huge which in turn generates more short lived Java objects. This cause pressure of GC.using unsafe and offheap will reduce the GC overhead.offheap can be accessed through java unsafe.hence enable.unsafe.in.query.procressing needs to be true</td>
+<td>Data that needs to be scanned in huge which in turn generates more short lived Java objects. This cause pressure of GC.using unsafe and offheap will reduce the GC overhead.offheap can be accessed through java unsafe.hence enable.unsafe.in.query.processing needs to be true</td>
</tr>
<tr>
<td>Query</td>
diff --git a/src/main/webapp/videogallery.html b/src/main/webapp/videogallery.html
index 359be97..614d8c6 100644
--- a/src/main/webapp/videogallery.html
+++ b/src/main/webapp/videogallery.html
@@ -49,6 +49,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.3/"
+ target="_blank">Apache CarbonData 1.5.3</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.2/"
target="_blank">Apache CarbonData 1.5.2</a></li>
<li>
@@ -60,12 +63,6 @@
<li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
target="_blank">Apache CarbonData 1.4.1</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a></li>
- <li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
- target="_blank">Apache CarbonData 1.3.1</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
target="_blank">Release Archive</a></li>
diff --git a/src/site/markdown/datamap-management.md b/src/site/markdown/datamap-management.md
index 0dc4718..087c70a 100644
--- a/src/site/markdown/datamap-management.md
+++ b/src/site/markdown/datamap-management.md
@@ -141,6 +141,7 @@ There is a SHOW DATAMAPS command, when this is issued, system will read all data
- DataMapName
- DataMapProviderName like mv, preaggreagte, timeseries, etc
- Associated Table
+- DataMap Properties
### Compaction on DataMap
diff --git a/src/site/markdown/ddl-of-carbondata.md b/src/site/markdown/ddl-of-carbondata.md
index 0d0e5bd..07a2670 100644
--- a/src/site/markdown/ddl-of-carbondata.md
+++ b/src/site/markdown/ddl-of-carbondata.md
@@ -67,6 +67,7 @@ CarbonData DDL statements are documented here,which includes:
* [SPLIT PARTITION](#split-a-partition)
* [DROP PARTITION](#drop-a-partition)
* [BUCKETING](#bucketing)
+* [CACHE](#cache)
## CREATE TABLE
@@ -1088,4 +1089,46 @@ Users can specify which columns to include and exclude for local dictionary gene
TBLPROPERTIES ('BUCKETNUMBER'='4', 'BUCKETCOLUMNS'='productName')
```
+## CACHE
+ CarbonData internally uses LRU caching to improve the performance. The user can get information
+ about current cache used status in memory through the following command:
+
+ ```sql
+ SHOW METACACHE
+ ```
+
+ This shows the overall memory consumed in the cache by categories - index files, dictionary and
+ datamaps. This also shows the cache usage by all the tables and children tables in the current
+ database.
+
+ ```sql
+ SHOW METACACHE ON TABLE tableName
+ ```
+
+ This shows detailed information on cache usage by the table `tableName` and its carbonindex files,
+ its dictionary files, its datamaps and children tables.
+
+ This command is not allowed on child tables.
+
+ ```sql
+ DROP METACACHE ON TABLE tableName
+ ```
+
+ This clears any entry in cache by the table `tableName`, its carbonindex files,
+ its dictionary files, its datamaps and children tables.
+
+ This command is not allowed on child tables.
+
+### Important points
+
+ 1. Cache information is updated only after the select query is executed.
+
+ 2. In case of alter table the already loaded cache is invalidated when any subsequent select query
+ is fired.
+
+ 3. Dictionary is loaded in cache only when the dictionary columns are queried upon. If we don't do
+ direct query on dictionary column, cache will not be loaded.
+ If we do `SELECT * FROM t1`, and even though for this case dictionary is loaded, it is loaded in
+ executor and not on driver, and the final result rows are returned back to driver, and thus will
+ produce no trace on driver cache if we do `SHOW METACACHE` or `SHOW METACACHE ON TABLE t1`.
diff --git a/src/site/markdown/dml-of-carbondata.md b/src/site/markdown/dml-of-carbondata.md
index f89c49a..6ec0520 100644
--- a/src/site/markdown/dml-of-carbondata.md
+++ b/src/site/markdown/dml-of-carbondata.md
@@ -35,10 +35,13 @@ CarbonData DML statements are documented here,which includes:
This command is used to load csv files to carbondata, OPTIONS are not mandatory for data loading process.
```
- LOAD DATA [LOCAL] INPATH 'folder_path'
+ LOAD DATA INPATH 'folder_path'
INTO TABLE [db_name.]table_name
OPTIONS(property_name=property_value, ...)
```
+ **NOTE**:
+ * Use 'file://' prefix to indicate local input files path, but it just supports local mode.
+ * If run on cluster mode, please upload all input files to distributed file system, for example 'hdfs://' for hdfs.
**Supported Properties:**
@@ -232,7 +235,7 @@ CarbonData DML statements are documented here,which includes:
Example:
```
- LOAD DATA local inpath '/opt/rawdata/data.csv' INTO table carbontable
+ LOAD DATA inpath '/opt/rawdata/data.csv' INTO table carbontable
options('DELIMITER'=',', 'QUOTECHAR'='"','COMMENTCHAR'='#',
'HEADER'='false',
'FILEHEADER'='empno,empname,designation,doj,workgroupcategory,
@@ -350,17 +353,19 @@ CarbonData DML statements are documented here,which includes:
This command allows you to load data using static partition.
```
- LOAD DATA [LOCAL] INPATH 'folder_path'
+ LOAD DATA INPATH 'folder_path'
INTO TABLE [db_name.]table_name PARTITION (partition_spec)
- OPTIONS(property_name=property_value, ...)
- INSERT INTO INTO TABLE [db_name.]table_name PARTITION (partition_spec) <SELECT STATEMENT>
+ OPTIONS(property_name=property_value, ...)
+
+ INSERT INTO TABLE [db_name.]table_name PARTITION (partition_spec) <SELECT STATEMENT>
```
Example:
```
- LOAD DATA LOCAL INPATH '${env:HOME}/staticinput.csv'
+ LOAD DATA INPATH '${env:HOME}/staticinput.csv'
INTO TABLE locationTable
- PARTITION (country = 'US', state = 'CA')
+ PARTITION (country = 'US', state = 'CA')
+
INSERT INTO TABLE locationTable
PARTITION (country = 'US', state = 'AL')
SELECT <columns list excluding partition columns> FROM another_user
@@ -372,8 +377,9 @@ CarbonData DML statements are documented here,which includes:
Example:
```
- LOAD DATA LOCAL INPATH '${env:HOME}/staticinput.csv'
- INTO TABLE locationTable
+ LOAD DATA INPATH '${env:HOME}/staticinput.csv'
+ INTO TABLE locationTable
+
INSERT INTO TABLE locationTable
SELECT <columns list excluding partition columns> FROM another_user
```
diff --git a/src/site/markdown/faq.md b/src/site/markdown/faq.md
index 7317d1c..9ba7082 100644
--- a/src/site/markdown/faq.md
+++ b/src/site/markdown/faq.md
@@ -43,6 +43,7 @@
- [Failed to insert data on the cluster](#failed-to-insert-data-on-the-cluster)
- [Failed to execute Concurrent Operations(Load,Insert,Update) on table by multiple workers](#failed-to-execute-concurrent-operations-on-table-by-multiple-workers)
- [Failed to create a table with a single numeric column](#failed-to-create-a-table-with-a-single-numeric-column)
+- [Failed to create datamap and drop datamap is also not working](#failed-to-create-datamap-and-drop-datamap-is-also-not-working)
##
@@ -474,4 +475,21 @@ Note : Refrain from using "mvn clean package" without specifying the profile.
A single column that can be considered as dimension is mandatory for table creation.
+## Failed to create datamap and drop datamap is also not working
+
+ **Symptom**
+
+ Execution fails with the following exception :
+
+ ```
+ HDFS Quota Exceeded
+ ```
+
+ **Possible Cause**
+
+ HDFS Quota is set, and it is not letting carbondata write or modify any files.
+
+ **Procedure**
+ Drop that particular datamap using Drop Table command using table name as
+ parentTableName_datamapName so as to clear the stale folders.
diff --git a/src/site/markdown/presto-guide.md b/src/site/markdown/presto-guide.md
index 6bb8196..c5bcd76 100644
--- a/src/site/markdown/presto-guide.md
+++ b/src/site/markdown/presto-guide.md
@@ -27,26 +27,37 @@ This tutorial provides a quick introduction to using current integration/presto
## Presto Multinode Cluster Setup for Carbondata
### Installing Presto
- 1. Download the 0.210 version of Presto using:
+To know about which version of presto is supported by this version of carbon, visit
+https://github.com/apache/carbondata/blob/master/integration/presto/pom.xml
+and look for ```<presto.version>```
+
+_Example:_
+ `<presto.version>0.217</presto.version>`
+This means current version of carbon supports presto 0.217 version.
+
+_Note:_
+Currently carbondata supports only one version of presto, cannot handle multiple versions at same time. If user wish to use older version of presto, then need to use older version of carbon (other old branches, say branch-1.5 and check the supported presto version in it's pom.xml file in integration/presto/)
+
+ 1. Download that version of Presto (say 0.217) using below command:
```
- wget https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.210/presto-server-0.210.tar.gz
+ wget https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.217/presto-server-0.217.tar.gz
```
- 2. Extract Presto tar file: `tar zxvf presto-server-0.210.tar.gz`.
+ 2. Extract Presto tar file: `tar zxvf presto-server-0.217.tar.gz`.
- 3. Download the Presto CLI for the coordinator and name it presto.
+ 3. Download the Presto CLI of the same presto server version (say 0.217) for the coordinator and name it presto.
```
- wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.210/presto-cli-0.210-executable.jar
+ wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.217/presto-cli-0.217-executable.jar
- mv presto-cli-0.210-executable.jar presto
+ mv presto-cli-0.217-executable.jar presto
chmod +x presto
```
### Create Configuration Files
- 1. Create `etc` folder in presto-server-0.210 directory.
+ 1. Create `etc` folder in presto-server-0.217 directory.
2. Create `config.properties`, `jvm.config`, `log.properties`, and `node.properties` files.
3. Install uuid to generate a node.id.
@@ -137,12 +148,12 @@ Then, `query.max-memory=<30GB * number of nodes>`.
### Start Presto Server on all nodes
```
-./presto-server-0.210/bin/launcher start
+./presto-server-0.217/bin/launcher start
```
To run it as a background process.
```
-./presto-server-0.210/bin/launcher run
+./presto-server-0.217/bin/launcher run
```
To run it in foreground.
@@ -165,7 +176,7 @@ Now you can use the Presto CLI on the coordinator to query data sources in the c
## Presto Single Node Setup for Carbondata
### Config presto server
-* Download presto server (0.210 is suggested and supported) : https://repo1.maven.org/maven2/com/facebook/presto/presto-server/
+* Download presto server (0.217 is suggested and supported) : https://repo1.maven.org/maven2/com/facebook/presto/presto-server/
* Finish presto configuration following https://prestodb.io/docs/current/installation/deployment.html.
A configuration example:
@@ -271,7 +282,7 @@ Load data statement in Spark can be used to create carbondata tables. And then y
carbondata files.
### Query carbondata in CLI of presto
-* Download presto cli client of version 0.210 : https://repo1.maven.org/maven2/com/facebook/presto/presto-cli
+* Download presto cli client of version 0.217 : https://repo1.maven.org/maven2/com/facebook/presto/presto-cli
* Start CLI:
diff --git a/src/site/markdown/quick-start-guide.md b/src/site/markdown/quick-start-guide.md
index 244a9ae..316fa26 100644
--- a/src/site/markdown/quick-start-guide.md
+++ b/src/site/markdown/quick-start-guide.md
@@ -241,7 +241,9 @@ mv carbondata.tar.gz carbonlib/
--executor-cores 2
```
-**NOTE**: Make sure you have permissions for CarbonData JARs and files through which driver and executor will start.
+**NOTE**:
+ - Make sure you have permissions for CarbonData JARs and files through which driver and executor will start.
+ - If use Spark + Hive 1.1.X, it needs to add carbondata assembly jar and carbondata-hive jar into parameter 'spark.sql.hive.metastore.jars' in spark-default.conf file.
@@ -485,4 +487,4 @@ select * from carbon_table;
**Note :** Create Tables and data loads should be done before executing queries as we can not create carbon table from this interface.
-```
\ No newline at end of file
+```
diff --git a/src/site/markdown/usecases.md b/src/site/markdown/usecases.md
index 0cfcf85..8ff4975 100644
--- a/src/site/markdown/usecases.md
+++ b/src/site/markdown/usecases.md
@@ -148,8 +148,8 @@ Use all columns are no-dictionary as the cardinality is high.
| Compaction | carbon.number.of.cores.while.compacting | 12 | Higher number of cores can improve the compaction speed.Data size is huge.Compaction need to use more threads to speed up the process |
| Compaction | carbon.enable.auto.load.merge | FALSE | Doing auto minor compaction is costly process as data size is huge.Perform manual compaction when the cluster is less loaded |
| Query | carbon.enable.vector.reader | true | To fetch results faster, supporting spark vector processing will speed up the query |
-| Query | enable.unsafe.in.query.procressing | true | Data that needs to be scanned in huge which in turn generates more short lived Java objects. This cause pressure of GC.using unsafe and offheap will reduce the GC overhead |
-| Query | use.offheap.in.query.processing | true | Data that needs to be scanned in huge which in turn generates more short lived Java objects. This cause pressure of GC.using unsafe and offheap will reduce the GC overhead.offheap can be accessed through java unsafe.hence enable.unsafe.in.query.procressing needs to be true |
+| Query | enable.unsafe.in.query.processing | true | Data that needs to be scanned in huge which in turn generates more short lived Java objects. This cause pressure of GC.using unsafe and offheap will reduce the GC overhead |
+| Query | use.offheap.in.query.processing | true | Data that needs to be scanned in huge which in turn generates more short lived Java objects. This cause pressure of GC.using unsafe and offheap will reduce the GC overhead.offheap can be accessed through java unsafe.hence enable.unsafe.in.query.processing needs to be true |
| Query | enable.unsafe.columnpage | TRUE | Keep the column pages in offheap memory so that the memory overhead due to java object is less and also reduces GC pressure. |
| Query | carbon.unsafe.working.memory.in.mb | 10240 | Amount of memory to use for offheap operations, you can increase this memory based on the data size |