You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@carbondata.apache.org by ch...@apache.org on 2017/04/27 07:01:07 UTC
[1/2] incubator-carbondata-site git commit: update md files
Repository: incubator-carbondata-site
Updated Branches:
refs/heads/asf-site 24e325ab0 -> b22227019
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/src/main/webapp/quick-start-guide.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/quick-start-guide.html b/src/main/webapp/quick-start-guide.html
index 0246f64..633bc6c 100644
--- a/src/main/webapp/quick-start-guide.html
+++ b/src/main/webapp/quick-start-guide.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/src/main/webapp/supported-data-types-in-carbondata.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/supported-data-types-in-carbondata.html b/src/main/webapp/supported-data-types-in-carbondata.html
index b56bc59..9757fb3 100644
--- a/src/main/webapp/supported-data-types-in-carbondata.html
+++ b/src/main/webapp/supported-data-types-in-carbondata.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/src/main/webapp/troubleshooting.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/troubleshooting.html b/src/main/webapp/troubleshooting.html
index 48f8871..7ae7dc7 100644
--- a/src/main/webapp/troubleshooting.html
+++ b/src/main/webapp/troubleshooting.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/src/main/webapp/useful-tips-on-carbondata.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/useful-tips-on-carbondata.html b/src/main/webapp/useful-tips-on-carbondata.html
index 39e6b3c..b5ff71c 100644
--- a/src/main/webapp/useful-tips-on-carbondata.html
+++ b/src/main/webapp/useful-tips-on-carbondata.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
@@ -162,7 +162,8 @@
The following sections will elaborate on the above topics :</p>
<ul>
<li><a href="#suggestions-to-create-carbondata-table">Suggestions to create CarbonData Table</a></li>
-<li><a href="#configurations-for-optimizing-carbondata-performance">Configurations For Optimizing CarbonData Performance</a></li>
+<li><a href="#configuration-for-optimizing-data-loading-performance-for-massive-data">Configuration for Optimizing Data Loading performance for Massive Data</a></li>
+<li><a href="#optimizing-mass-data-loading">Optimizing Mass Data Loading</a></li>
</ul>
<h2>
<a id="suggestions-to-create-carbondata-table" class="anchor" href="#suggestions-to-create-carbondata-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Suggestions to Create CarbonData Table</h2>
@@ -236,7 +237,7 @@ The create table command can be modified as suggested below :</p>
<pre><code> create table carbondata_table(
msisdn String,
...
- )STORED BY 'org.apache.carbondata.format'
+ )STORED BY 'org.apache.carbondata.format'
TBLPROPERTIES ( 'DICTIONARY_EXCLUDE'='MSISDN,..',
'DICTIONARY_INCLUDE'='...');
</code></pre>
@@ -257,7 +258,7 @@ The create table command can be modified as suggested below :</p>
HOST String,
MSISDN String,
...
- )STORED BY 'org.apache.carbondata.format'
+ )STORED BY 'org.apache.carbondata.format'
TBLPROPERTIES ( 'DICTIONARY_EXCLUDE'='MSISDN,HOST..',
'DICTIONARY_INCLUDE'='Dime_1..');
</code></pre>
@@ -274,7 +275,7 @@ The create table command can be modified as below :</p>
HOST String,
MSISDN String,
...
- )STORED BY 'org.apache.carbondata.format'
+ )STORED BY 'org.apache.carbondata.format'
TBLPROPERTIES ( 'DICTIONARY_EXCLUDE'='MSISDN,HOST,IMSI..',
'DICTIONARY_INCLUDE'='Dime_1,END_TIME,BEGIN_TIME..');
</code></pre>
@@ -294,7 +295,7 @@ query performance. The create table command can be modified as below :</p>
counter_2 double,
...
counter_100 double
- )STORED BY 'org.apache.carbondata.format'
+ )STORED BY 'org.apache.carbondata.format'
TBLPROPERTIES ( 'DICTIONARY_EXCLUDE'='MSISDN,HOST,IMSI',
'DICTIONARY_INCLUDE'='Dime_1,END_TIME,BEGIN_TIME');
</code></pre>
@@ -316,9 +317,9 @@ suggested to put start_time at the end of dimensions.</p>
BEGIN_TIME bigint,
...
counter_100 double
- )STORED BY 'org.apache.carbondata.format'
+ )STORED BY 'org.apache.carbondata.format'
TBLPROPERTIES ( 'DICTIONARY_EXCLUDE'='MSISDN,HOST,IMSI',
- 'DICTIONARY_INCLUDE'='Dime_1,END_TIME,BEGIN_TIME');
+ 'DICTIONARY_INCLUDE'='Dime_1,END_TIME,BEGIN_TIME');
</code></pre>
<ul>
<li>
@@ -331,6 +332,61 @@ excessive memory usage.</p>
</li>
</ul>
<h2>
+<a id="configuration-for-optimizing-data-loading-performance-for-massive-data" class="anchor" href="#configuration-for-optimizing-data-loading-performance-for-massive-data" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Configuration for Optimizing Data Loading performance for Massive Data</h2>
+<p>CarbonData supports large data load, in this process sorting data while loading consumes a lot of memory and disk IO and
+this can result sometimes in "Out Of Memory" exception.
+If you do not have much memory to use, then you may prefer to slow the speed of data loading instead of data load failure.
+You can configure CarbonData by tuning following properties in carbon.properties file to get a better performance.:</p>
+<table>
+<thead>
+<tr>
+<th>Parameter</th>
+<th>Default Value</th>
+<th>Description/Tuning</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>carbon.number.of.cores.while.loading</td>
+<td>Default: 2.This value should be >= 2</td>
+<td>Specifies the number of cores used for data processing during data loading in CarbonData.</td>
+</tr>
+<tr>
+<td>carbon.sort.size</td>
+<td>Data loading</td>
+<td>Default: 100000. The value should be >= 100.</td>
+</tr>
+<tr>
+<td>carbon.sort.file.write.buffer.size</td>
+<td>Default: 50000.</td>
+<td>DataOutputStream buffer.</td>
+</tr>
+<tr>
+<td>carbon.number.of.cores.block.sort</td>
+<td>Default: 7</td>
+<td>If you have huge memory and cpus, increase it as you will</td>
+</tr>
+<tr>
+<td>carbon.merge.sort.reader.thread</td>
+<td>Default: 3</td>
+<td>Specifies the number of cores used for temp file merging during data loading in CarbonData.</td>
+</tr>
+<tr>
+<td>carbon.merge.sort.prefetch</td>
+<td>Default: true</td>
+<td>You may want set this value to false if you have not enough memory</td>
+</tr>
+</tbody>
+</table>
+<p>For example, if there are 10 million records ,and i have only 16 cores ,64GB memory, will be loaded to CarbonData table.
+Using the default configuration always fail in sort step. Modify carbon.properties as suggested below</p>
+<pre><code>carbon.number.of.cores.block.sort=1
+carbon.merge.sort.reader.thread=1
+carbon.sort.size=5000
+carbon.sort.file.write.buffer.size=5000
+carbon.merge.sort.prefetch=false
+</code></pre>
+<h2>
<a id="configurations-for-optimizing-carbondata-performance" class="anchor" href="#configurations-for-optimizing-carbondata-performance" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Configurations for Optimizing CarbonData Performance</h2>
<p>Recently we did some performance POC on CarbonData for Finance and telecommunication Field. It involved detailed queries and aggregation
scenarios. After the completion of POC, some of the configurations impacting the performance have been identified and tabulated below :</p>
@@ -396,6 +452,8 @@ scenarios. After the completion of POC, some of the configurations impacting the
</tr>
</tbody>
</table>
+<p>Note: If your CarbonData instance is provided only for query, you may specify the conf 'spark.speculation=true' which is conf
+in spark.</p>
</div>
</div>
</div>
[2/2] incubator-carbondata-site git commit: update md files
Posted by ch...@apache.org.
update md files
Project: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/commit/b2222701
Tree: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/tree/b2222701
Diff: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/diff/b2222701
Branch: refs/heads/asf-site
Commit: b22227019ae0370a2ad773d779e2353e7ac57e19
Parents: 24e325a
Author: chenliang613 <ch...@huawei.com>
Authored: Thu Apr 27 15:00:57 2017 +0800
Committer: chenliang613 <ch...@huawei.com>
Committed: Thu Apr 27 15:00:57 2017 +0800
----------------------------------------------------------------------
content/configuration-parameters.html | 6 +-
content/data-management.html | 16 +-
content/ddl-operation-on-carbondata.html | 313 +++++++++++++++----
content/dml-operation-on-carbondata.html | 6 +-
content/faq.html | 49 ++-
content/file-structure-of-carbondata.html | 6 +-
content/installation-guide.html | 8 +-
content/quick-start-guide.html | 6 +-
content/supported-data-types-in-carbondata.html | 6 +-
content/troubleshooting.html | 6 +-
content/useful-tips-on-carbondata.html | 78 ++++-
src/main/webapp/configuration-parameters.html | 6 +-
src/main/webapp/data-management.html | 16 +-
.../webapp/ddl-operation-on-carbondata.html | 313 +++++++++++++++----
.../webapp/dml-operation-on-carbondata.html | 6 +-
src/main/webapp/faq.html | 49 ++-
.../webapp/file-structure-of-carbondata.html | 6 +-
src/main/webapp/installation-guide.html | 8 +-
src/main/webapp/quick-start-guide.html | 6 +-
.../supported-data-types-in-carbondata.html | 6 +-
src/main/webapp/troubleshooting.html | 6 +-
src/main/webapp/useful-tips-on-carbondata.html | 78 ++++-
22 files changed, 792 insertions(+), 208 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/content/configuration-parameters.html
----------------------------------------------------------------------
diff --git a/content/configuration-parameters.html b/content/configuration-parameters.html
index 3b2d562..63943eb 100644
--- a/content/configuration-parameters.html
+++ b/content/configuration-parameters.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/content/data-management.html
----------------------------------------------------------------------
diff --git a/content/data-management.html b/content/data-management.html
index 0ae1ef8..b3a9c99 100644
--- a/content/data-management.html
+++ b/content/data-management.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
@@ -201,12 +201,12 @@ CarbonData store metadata. The table below lists the possible load status.</p>
</tr>
</tbody>
</table>
-<p>In case of failure, the error will be logged in error log. Details of loads can be seen with <a href="dml-operation-on-carbondata.html">SHOW SEGMENTS</a> command. The show segment command output consists of :</p>
+<p>In case of failure, the error will be logged in error log. Details of loads can be seen with <a href="dml-operation-on-carbondata.html#show-segments">SHOW SEGMENTS</a> command. The show segment command output consists of :</p>
<ul>
-<li>SegmentSequenceID</li>
-<li>START_TIME OF LOAD</li>
-<li>END_TIME OF LOAD</li>
-<li>LOAD STATUS</li>
+<li>SegmentSequenceId</li>
+<li>Status</li>
+<li>Load Start Time</li>
+<li>Load End Time</li>
</ul>
<p>The latest load will be displayed first in the output.</p>
<p>Refer to <a href="dml-operation-on-carbondata.html">DML operations on CarbonData</a> for load commands.</p>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/content/ddl-operation-on-carbondata.html
----------------------------------------------------------------------
diff --git a/content/ddl-operation-on-carbondata.html b/content/ddl-operation-on-carbondata.html
index a12d15c..81e8664 100644
--- a/content/ddl-operation-on-carbondata.html
+++ b/content/ddl-operation-on-carbondata.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
@@ -165,10 +165,18 @@
<ul>
<li><a href="#create-table">CREATE TABLE</a></li>
<li><a href="#show-table">SHOW TABLE</a></li>
+<li>
+<a href="#alter-table">ALTER TABLE</a>
+<ul>
+<li><a href="#rename-table">RENAME TABLE</a></li>
+<li><a href="#add-column">ADD COLUMN</a></li>
+<li><a href="#drop-columns">DROP COLUMNS</a></li>
+<li><a href="#change-data-type">CHANGE DATA TYPE</a></li>
+</ul>
+</li>
<li><a href="#drop-table">DROP TABLE</a></li>
<li><a href="#compaction">COMPACTION</a></li>
<li><a href="#bucketing">BUCKETING</a></li>
-<li><a href="#table-rename">TABLE RENAME</a></li>
</ul>
<h2>
<a id="create-table" class="anchor" href="#create-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CREATE TABLE</h2>
@@ -236,8 +244,8 @@
<p>Column groups with more than one column are stored in row format, instead of columnar format. By default, each column is a separate column group.</p>
</li>
</ul>
-<pre><code>TBLPROPERTIES ('COLUMN_GROUPS'='(column1, column3),
-(Column4,Column5,Column6)')
+<pre><code>TBLPROPERTIES ('COLUMN_GROUPS'='(column1, column2),
+(Column3,Column4,Column5)')
</code></pre>
<ul>
<li>
@@ -313,12 +321,239 @@ By default inverted index is enabled. The user can disable the inverted index cr
<pre><code> SHOW TABLES IN ProductSchema;
</code></pre>
<h2>
+<a id="alter-table" class="anchor" href="#alter-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>ALTER TABLE</h2>
+<p>The following section shall discuss the commands to modify the physical or logical state of the existing table(s).</p>
+<h3>
+<a id="rename-table" class="anchor" href="#rename-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a><strong>RENAME TABLE</strong>
+</h3>
+<p>This command is used to rename the existing table.</p>
+<pre><code> ALTER TABLE [db_name.]table_name RENAME TO new_table_name;
+</code></pre>
+<h4>
+<a id="parameter-description-2" class="anchor" href="#parameter-description-2" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h4>
+<table>
+<thead>
+<tr>
+<th>Parameter</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>db_Name</td>
+<td>Name of the database. If this parameter is left unspecified, the current database is selected.</td>
+</tr>
+<tr>
+<td>table_name</td>
+<td>Name of the existing table.</td>
+</tr>
+<tr>
+<td>new_table_name</td>
+<td>New table name for the existing table.</td>
+</tr>
+</tbody>
+</table>
+<h4>
+<a id="usage-guidelines-1" class="anchor" href="#usage-guidelines-1" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h4>
+<ul>
+<li>
+<p>Queries that require the formation of path using the table name for reading carbon store files, running in parallel with Rename command might fail during the renaming operation.</p>
+</li>
+<li>
+<p>Renaming of Secondary index table(s) is not permitted.</p>
+</li>
+</ul>
+<h4>
+<a id="examples" class="anchor" href="#examples" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Examples:</h4>
+<pre><code> ALTER TABLE carbon RENAME TO carbondata;
+</code></pre>
+<pre><code> ALTER TABLE test_db.carbon RENAME TO test_db.carbondata;
+</code></pre>
+<h3>
+<a id="add-column" class="anchor" href="#add-column" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a><strong>ADD COLUMN</strong>
+</h3>
+<p>This command is used to add a new column to the existing table.</p>
+<pre><code> ALTER TABLE [db_name.]table_name ADD COLUMNS (col_name data_type,...)
+ TBLPROPERTIES('DICTIONARY_INCLUDE'='col_name,...',
+ 'DICTIONARY_EXCLUDE'='col_name,...',
+ 'DEFAULT.VALUE.COLUMN_NAME'='default_value');
+</code></pre>
+<h4>
+<a id="parameter-description-3" class="anchor" href="#parameter-description-3" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h4>
+<table>
+<thead>
+<tr>
+<th>Parameter</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>db_Name</td>
+<td>Name of the database. If this parameter is left unspecified, the current database is selected.</td>
+</tr>
+<tr>
+<td>table_name</td>
+<td>Name of the existing table.</td>
+</tr>
+<tr>
+<td>col_name data_type</td>
+<td>Name of comma-separated column with data type. Column names contain letters, digits, and underscores (_).</td>
+</tr>
+</tbody>
+</table>
+<p>NOTE: Do not name the column after name, tupleId, PositionId, and PositionReference when creating Carbon tables because they are used internally by UPDATE, DELETE, and secondary index.</p>
+<h4>
+<a id="usage-guidelines-2" class="anchor" href="#usage-guidelines-2" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h4>
+<ul>
+<li>
+<p>Apart from DICTIONARY_INCLUDE, DICTIONARY_EXCLUDE and default_value no other property will be read. If any other property name is specified, error will not be thrown, it will be ignored.</p>
+</li>
+<li>
+<p>If default value is not specified, then NULL will be considered as the default value for the column.</p>
+</li>
+<li>
+<p>For addition of column, if DICTIONARY_INCLUDE and DICTIONARY_EXCLUDE are not specified, then the decision will be taken based on data type of the column.</p>
+</li>
+</ul>
+<h4>
+<a id="examples-1" class="anchor" href="#examples-1" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Examples:</h4>
+<pre><code> ALTER TABLE carbon ADD COLUMNS (a1 INT, b1 STRING);
+</code></pre>
+<pre><code> ALTER TABLE carbon ADD COLUMNS (a1 INT, b1 STRING) TBLPROPERTIES('DICTIONARY_EXCLUDE'='b1');
+</code></pre>
+<pre><code> ALTER TABLE carbon ADD COLUMNS (a1 INT, b1 STRING) TBLPROPERTIES('DICTIONARY_INCLUDE'='a1');
+</code></pre>
+<pre><code> ALTER TABLE carbon ADD COLUMNS (a1 INT, b1 STRING) TBLPROPERTIES('DEFAULT.VALUE.a1'='10');
+</code></pre>
+<h3>
+<a id="drop-columns" class="anchor" href="#drop-columns" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a><strong>DROP COLUMNS</strong>
+</h3>
+<p>This command is used to delete a existing column or multiple columns in a table.</p>
+<pre><code> ALTER TABLE [db_name.]table_name DROP COLUMNS (col_name, ...);
+</code></pre>
+<h4>
+<a id="parameter-description-4" class="anchor" href="#parameter-description-4" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h4>
+<table>
+<thead>
+<tr>
+<th>Parameter</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>db_Name</td>
+<td>Name of the database. If this parameter is left unspecified, the current database is selected.</td>
+</tr>
+<tr>
+<td>table_name</td>
+<td>Name of the existing table.</td>
+</tr>
+<tr>
+<td>col_name</td>
+<td>Name of comma-separated column with data type. Column names contain letters, digits, and underscores (_)</td>
+</tr>
+</tbody>
+</table>
+<h4>
+<a id="usage-guidelines-3" class="anchor" href="#usage-guidelines-3" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h4>
+<ul>
+<li>
+<p>Deleting a column will also clear the dictionary files, provided the column is of type dictionary.</p>
+</li>
+<li>
+<p>For delete column operation, there should be at least one key column that exists in the schema after deletion else error message will be displayed and the operation shall fail.</p>
+</li>
+</ul>
+<h4>
+<a id="examples-2" class="anchor" href="#examples-2" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Examples:</h4>
+<p>If the table contains 4 columns namely a1, b1, c1, and d1.</p>
+<ul>
+<li><strong>To delete a single column:</strong></li>
+</ul>
+<pre><code> ALTER TABLE carbon DROP COLUMNS (b1);
+</code></pre>
+<pre><code> ALTER TABLE test_db.carbon DROP COLUMNS (b1);
+</code></pre>
+<ul>
+<li><strong>To delete multiple columns:</strong></li>
+</ul>
+<pre><code> ALTER TABLE carbon DROP COLUMNS (b1,c1);
+</code></pre>
+<pre><code> ALTER TABLE carbon DROP COLUMNS (b1,c1);
+</code></pre>
+<h3>
+<a id="change-data-type" class="anchor" href="#change-data-type" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a><strong>CHANGE DATA TYPE</strong>
+</h3>
+<p>This command is used to change the data type from INT to BIGINT or decimal precision from lower to higher.</p>
+<pre><code> ALTER TABLE [db_name.]table_name CHANGE col_name col_name changed_column_type;
+</code></pre>
+<h4>
+<a id="parameter-description-5" class="anchor" href="#parameter-description-5" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h4>
+<table>
+<thead>
+<tr>
+<th>Parameter</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>db_Name</td>
+<td>Name of the database. If this parameter is left unspecified, the current database is selected.</td>
+</tr>
+<tr>
+<td>table_name</td>
+<td>Name of the existing table.</td>
+</tr>
+<tr>
+<td>col_name</td>
+<td>Name of comma-separated column with data type. Column names contain letters, digits, and underscores (_).</td>
+</tr>
+<tr>
+<td>changed_column_type</td>
+<td>The change in the data type.</td>
+</tr>
+</tbody>
+</table>
+<h4>
+<a id="usage-guidelines-4" class="anchor" href="#usage-guidelines-4" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h4>
+<ul>
+<li>Change of decimal data type from lower precision to higher precision will only be supported for cases where there is no data loss.</li>
+</ul>
+<h4>
+<a id="valid-scenarios" class="anchor" href="#valid-scenarios" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Valid Scenarios</h4>
+<ul>
+<li>
+<p>Invalid scenario - Change of decimal precision from (10,2) to (10,5) is not valid as in this case only scale is increased but total number of digits remain the same.</p>
+</li>
+<li>
+<p>Valid scenario - Change of decimal precision from (10,2) to (12,3) is valid as the total number of digits are increased by 2 but scale is increased only by 1 which will not lead to any data loss.</p>
+</li>
+<li>
+<p>Note :The allowed range is 38,38 (precision, scale) and is a valid upper case scenario which is not resulting in data loss.</p>
+</li>
+</ul>
+<h4>
+<a id="examples-3" class="anchor" href="#examples-3" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Examples:</h4>
+<ul>
+<li><strong>Changing data type of column a1 from INT to BIGINT</strong></li>
+</ul>
+<pre><code> ALTER TABLE test_db.carbon CHANGE a1 a1 BIGINT;
+</code></pre>
+<ul>
+<li><strong>Changing decimal precision of column a1 from 10 to 18.</strong></li>
+</ul>
+<pre><code> ALTER TABLE test_db.carbon CHANGE a1 a1 DECIMAL(18,2);
+</code></pre>
+<h2>
<a id="drop-table" class="anchor" href="#drop-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>DROP TABLE</h2>
<p>This command is used to delete an existing table.</p>
<pre><code> DROP TABLE [IF EXISTS] [db_name.]table_name;
</code></pre>
<h3>
-<a id="parameter-description-2" class="anchor" href="#parameter-description-2" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h3>
+<a id="parameter-description-6" class="anchor" href="#parameter-description-6" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h3>
<table>
<thead>
<tr>
@@ -351,7 +586,7 @@ By default inverted index is enabled. The user can disable the inverted index cr
</code></pre>
<p>To get details about Compaction refer to <a href="data-management.html">Data Management</a></p>
<h3>
-<a id="parameter-description-3" class="anchor" href="#parameter-description-3" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h3>
+<a id="parameter-description-7" class="anchor" href="#parameter-description-7" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h3>
<table>
<thead>
<tr>
@@ -397,8 +632,8 @@ of columns is used.</p>
TBLPROPERTIES('BUCKETNUMBER'='noOfBuckets',
'BUCKETCOLUMNS'='columnname')
</code></pre>
-<h2>
-<a id="parameter-description-4" class="anchor" href="#parameter-description-4" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h2>
+<h3>
+<a id="parameter-description-8" class="anchor" href="#parameter-description-8" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h3>
<table>
<thead>
<tr>
@@ -420,8 +655,8 @@ of columns is used.</p>
</tr>
</tbody>
</table>
-<h2>
-<a id="usage-guidelines-1" class="anchor" href="#usage-guidelines-1" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h2>
+<h3>
+<a id="usage-guidelines-5" class="anchor" href="#usage-guidelines-5" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h3>
<ul>
<li>
<p>The feature is supported for Spark 1.6.2 onwards, but the performance optimization is evident from Spark 2.1 onwards.</p>
@@ -433,69 +668,25 @@ of columns is used.</p>
<p>Columns in the BUCKETCOLUMN parameter must be only dimension. The BUCKETCOLUMN parameter can not be a measure or a combination of measures and dimensions.</p>
</li>
</ul>
-<h2>
-<a id="example-" class="anchor" href="#example-" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Example :</h2>
+<h3>
+<a id="example-" class="anchor" href="#example-" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Example :</h3>
<pre><code> CREATE TABLE IF NOT EXISTS productSchema.productSalesTable (
productNumber Int,
+ saleQuantity Int,
productName String,
storeCity String,
storeProvince String,
productCategory String,
productBatch String,
- saleQuantity Int,
revenue Int)
STORED BY 'carbondata'
- TBLPROPERTIES ('COLUMN_GROUPS'='(productName,productNumber)',
+ TBLPROPERTIES ('COLUMN_GROUPS'='(productNumber,saleQuantity)',
'DICTIONARY_EXCLUDE'='productName',
- 'DICTIONARY_INCLUDE'='productNumber',
+ 'DICTIONARY_INCLUDE'='productNumber,saleQuantity',
'NO_INVERTED_INDEX'='productBatch',
'BUCKETNUMBER'='4',
'BUCKETCOLUMNS'='productName')
</code></pre>
-<h2>
-<a id="table-rename" class="anchor" href="#table-rename" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>TABLE RENAME</h2>
-<p>This command is used to rename the existing table.</p>
-<h3>
-<a id="syntax-1" class="anchor" href="#syntax-1" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Syntax</h3>
-<pre><code> ALTER TABLE [db_name.]table_name RENAME TO new_table_name;
-</code></pre>
-<h3>
-<a id="parameter-description-5" class="anchor" href="#parameter-description-5" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h3>
-<table>
-<thead>
-<tr>
-<th>Parameter</th>
-<th>Description</th>
-</tr>
-</thead>
-<tbody>
-<tr>
-<td>db_name</td>
-<td>Name of the database. If this parameter is left unspecified, the current database is selected.</td>
-</tr>
-<tr>
-<td>table_name</td>
-<td>Name of the existing table.</td>
-</tr>
-<tr>
-<td>new_table_name</td>
-<td>New table name for the existing table.</td>
-</tr>
-</tbody>
-</table>
-<h3>
-<a id="usage-guidelines-2" class="anchor" href="#usage-guidelines-2" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h3>
-<p>Following conditions must be met for successful rename operation:</p>
-<ul>
-<li>Queries running in parallel which requires the formation of path using the table name for reading carbon store files might fail during this operation.</li>
-<li>Secondary index table rename is not permitted.</li>
-</ul>
-<h3>
-<a id="example-3" class="anchor" href="#example-3" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Example:</h3>
-<pre><code> ALTER TABLE carbon RENAME TO carbondata;
-
- ALTER TABLE test_db.carbon RENAME TO test_db.carbondata;
-</code></pre>
</div>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/content/dml-operation-on-carbondata.html
----------------------------------------------------------------------
diff --git a/content/dml-operation-on-carbondata.html b/content/dml-operation-on-carbondata.html
index 52a70db..e0c234e 100644
--- a/content/dml-operation-on-carbondata.html
+++ b/content/dml-operation-on-carbondata.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/content/faq.html
----------------------------------------------------------------------
diff --git a/content/faq.html b/content/faq.html
index 8567346..54fcc09 100644
--- a/content/faq.html
+++ b/content/faq.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
@@ -166,6 +166,7 @@
<li><a href="#how-to-specify-store-location-while-creating-carbon-session">How to specify store location while creating carbon session?</a></li>
<li><a href="#what-is-carbon-lock-type">What is Carbon Lock Type?</a></li>
<li><a href="#how-to-resolve-abstract-method-error">How to resolve Abstract Method Error?</a></li>
+<li><a href="#how-carbon-will-behave-when-execute-insert-operation-in-abnormal-scenarios">How Carbon will behave when execute insert operation in abnormal scenarios?</a></li>
</ul>
<h2>
<a id="what-are-bad-records" class="anchor" href="#what-are-bad-records" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>What are Bad Records?</h2>
@@ -214,6 +215,48 @@ The property carbon.lock.type configuration specifies the type of lock to be acq
<h2>
<a id="how-to-resolve-abstract-method-error" class="anchor" href="#how-to-resolve-abstract-method-error" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>How to resolve Abstract Method Error?</h2>
<p>In order to build CarbonData project it is necessary to specify the spark profile. The spark profile sets the Spark Version. You need to specify the <code>spark version</code> while using Maven to build project.</p>
+<h2>
+<a id="how-carbon-will-behave-when-execute-insert-operation-in-abnormal-scenarios" class="anchor" href="#how-carbon-will-behave-when-execute-insert-operation-in-abnormal-scenarios" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>How Carbon will behave when execute insert operation in abnormal scenarios?</h2>
+<p>Carbon support insert operation, you can refer to the syntax mentioned in <a href="http://carbondata.apache.org/dml-operation-on-carbondata" target=_blank>DML Operations on CarbonData</a>.
+First, create a soucre table in spark-sql and load data into this created table.</p>
+<pre><code>CREATE TABLE source_table(
+id String,
+name String,
+city String)
+ROW FORMAT DELIMITED FIELDS TERMINATED BY ",";
+</code></pre>
+<pre><code>SELECT * FROM source_table;
+id name city
+1 jack beijing
+2 erlu hangzhou
+3 davi shenzhen
+</code></pre>
+<p><strong>Scenario 1</strong> :</p>
+<p>Suppose, the column order in carbon table is different from source table, use script "SELECT * FROM carbon table" to query, will get the column order similar as source table, rather than in carbon table's column order as expected.</p>
+<pre><code>CREATE TABLE IF NOT EXISTS carbon_table(
+id String,
+city String,
+name String)
+STORED BY 'carbondata';
+</code></pre>
+<pre><code>INSERT INTO TABLE carbon_table SELECT * FROM source_table;
+</code></pre>
+<pre><code>SELECT * FROM carbon_table;
+id city name
+1 jack beijing
+2 erlu hangzhou
+3 davi shenzhen
+</code></pre>
+<p>As result shows, the second column is city in carbon table, but what inside is name, such as jack. This phenomenon is same with insert data into hive table.</p>
+<p>If you want to insert data into corresponding column in carbon table, you have to specify the column order same in insert statment.</p>
+<pre><code>INSERT INTO TABLE carbon_table SELECT id, city, name FROM source_table;
+</code></pre>
+<p><strong>Scenario 2</strong> :</p>
+<p>Insert operation will be failed when the number of column in carbon table is different from the column specified in select statement. The following insert operation will be failed.</p>
+<pre><code>INSERT INTO TABLE carbon_table SELECT id, city FROM source_table;
+</code></pre>
+<p><strong>Scenario 3</strong> :</p>
+<p>When the column type in carbon table is different from from the column specified in select statement. The insert operation will still success, but you may get NULL in result, because NULL will be substitute value when conversion type failed.</p>
</div>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/content/file-structure-of-carbondata.html
----------------------------------------------------------------------
diff --git a/content/file-structure-of-carbondata.html b/content/file-structure-of-carbondata.html
index 564486b..985b116 100644
--- a/content/file-structure-of-carbondata.html
+++ b/content/file-structure-of-carbondata.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/content/installation-guide.html
----------------------------------------------------------------------
diff --git a/content/installation-guide.html b/content/installation-guide.html
index 717ca62..fd31349 100644
--- a/content/installation-guide.html
+++ b/content/installation-guide.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
@@ -324,7 +324,7 @@ mv carbondata.tar.gz carbonlib/
<tr>
<td>spark.driver.extraClassPath</td>
<td>Extra classpath entries to prepend to the classpath of the driver. <strong>NOTE</strong>: If SPARK_CLASSPATH is defined in spark-env.sh, then comment it and append the value in below parameter spark.driver.extraClassPath.</td>
-<td><code>$SPARK_HOME/carbonlib/carbonlib/*</code></td>
+<td><code>$SPARK_HOME/carbonlib/*</code></td>
</tr>
<tr>
<td>spark.driver.extraJavaOptions</td>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/content/quick-start-guide.html
----------------------------------------------------------------------
diff --git a/content/quick-start-guide.html b/content/quick-start-guide.html
index 0246f64..633bc6c 100644
--- a/content/quick-start-guide.html
+++ b/content/quick-start-guide.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/content/supported-data-types-in-carbondata.html
----------------------------------------------------------------------
diff --git a/content/supported-data-types-in-carbondata.html b/content/supported-data-types-in-carbondata.html
index b56bc59..9757fb3 100644
--- a/content/supported-data-types-in-carbondata.html
+++ b/content/supported-data-types-in-carbondata.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/content/troubleshooting.html
----------------------------------------------------------------------
diff --git a/content/troubleshooting.html b/content/troubleshooting.html
index 48f8871..7ae7dc7 100644
--- a/content/troubleshooting.html
+++ b/content/troubleshooting.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/content/useful-tips-on-carbondata.html
----------------------------------------------------------------------
diff --git a/content/useful-tips-on-carbondata.html b/content/useful-tips-on-carbondata.html
index 39e6b3c..b5ff71c 100644
--- a/content/useful-tips-on-carbondata.html
+++ b/content/useful-tips-on-carbondata.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
@@ -162,7 +162,8 @@
The following sections will elaborate on the above topics :</p>
<ul>
<li><a href="#suggestions-to-create-carbondata-table">Suggestions to create CarbonData Table</a></li>
-<li><a href="#configurations-for-optimizing-carbondata-performance">Configurations For Optimizing CarbonData Performance</a></li>
+<li><a href="#configuration-for-optimizing-data-loading-performance-for-massive-data">Configuration for Optimizing Data Loading performance for Massive Data</a></li>
+<li><a href="#optimizing-mass-data-loading">Optimizing Mass Data Loading</a></li>
</ul>
<h2>
<a id="suggestions-to-create-carbondata-table" class="anchor" href="#suggestions-to-create-carbondata-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Suggestions to Create CarbonData Table</h2>
@@ -236,7 +237,7 @@ The create table command can be modified as suggested below :</p>
<pre><code> create table carbondata_table(
msisdn String,
...
- )STORED BY 'org.apache.carbondata.format'
+ )STORED BY 'org.apache.carbondata.format'
TBLPROPERTIES ( 'DICTIONARY_EXCLUDE'='MSISDN,..',
'DICTIONARY_INCLUDE'='...');
</code></pre>
@@ -257,7 +258,7 @@ The create table command can be modified as suggested below :</p>
HOST String,
MSISDN String,
...
- )STORED BY 'org.apache.carbondata.format'
+ )STORED BY 'org.apache.carbondata.format'
TBLPROPERTIES ( 'DICTIONARY_EXCLUDE'='MSISDN,HOST..',
'DICTIONARY_INCLUDE'='Dime_1..');
</code></pre>
@@ -274,7 +275,7 @@ The create table command can be modified as below :</p>
HOST String,
MSISDN String,
...
- )STORED BY 'org.apache.carbondata.format'
+ )STORED BY 'org.apache.carbondata.format'
TBLPROPERTIES ( 'DICTIONARY_EXCLUDE'='MSISDN,HOST,IMSI..',
'DICTIONARY_INCLUDE'='Dime_1,END_TIME,BEGIN_TIME..');
</code></pre>
@@ -294,7 +295,7 @@ query performance. The create table command can be modified as below :</p>
counter_2 double,
...
counter_100 double
- )STORED BY 'org.apache.carbondata.format'
+ )STORED BY 'org.apache.carbondata.format'
TBLPROPERTIES ( 'DICTIONARY_EXCLUDE'='MSISDN,HOST,IMSI',
'DICTIONARY_INCLUDE'='Dime_1,END_TIME,BEGIN_TIME');
</code></pre>
@@ -316,9 +317,9 @@ suggested to put start_time at the end of dimensions.</p>
BEGIN_TIME bigint,
...
counter_100 double
- )STORED BY 'org.apache.carbondata.format'
+ )STORED BY 'org.apache.carbondata.format'
TBLPROPERTIES ( 'DICTIONARY_EXCLUDE'='MSISDN,HOST,IMSI',
- 'DICTIONARY_INCLUDE'='Dime_1,END_TIME,BEGIN_TIME');
+ 'DICTIONARY_INCLUDE'='Dime_1,END_TIME,BEGIN_TIME');
</code></pre>
<ul>
<li>
@@ -331,6 +332,61 @@ excessive memory usage.</p>
</li>
</ul>
<h2>
+<a id="configuration-for-optimizing-data-loading-performance-for-massive-data" class="anchor" href="#configuration-for-optimizing-data-loading-performance-for-massive-data" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Configuration for Optimizing Data Loading performance for Massive Data</h2>
+<p>CarbonData supports large data load, in this process sorting data while loading consumes a lot of memory and disk IO and
+this can result sometimes in "Out Of Memory" exception.
+If you do not have much memory to use, then you may prefer to slow the speed of data loading instead of data load failure.
+You can configure CarbonData by tuning following properties in carbon.properties file to get a better performance.:</p>
+<table>
+<thead>
+<tr>
+<th>Parameter</th>
+<th>Default Value</th>
+<th>Description/Tuning</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>carbon.number.of.cores.while.loading</td>
+<td>Default: 2.This value should be >= 2</td>
+<td>Specifies the number of cores used for data processing during data loading in CarbonData.</td>
+</tr>
+<tr>
+<td>carbon.sort.size</td>
+<td>Data loading</td>
+<td>Default: 100000. The value should be >= 100.</td>
+</tr>
+<tr>
+<td>carbon.sort.file.write.buffer.size</td>
+<td>Default: 50000.</td>
+<td>DataOutputStream buffer.</td>
+</tr>
+<tr>
+<td>carbon.number.of.cores.block.sort</td>
+<td>Default: 7</td>
+<td>If you have huge memory and cpus, increase it as you will</td>
+</tr>
+<tr>
+<td>carbon.merge.sort.reader.thread</td>
+<td>Default: 3</td>
+<td>Specifies the number of cores used for temp file merging during data loading in CarbonData.</td>
+</tr>
+<tr>
+<td>carbon.merge.sort.prefetch</td>
+<td>Default: true</td>
+<td>You may want set this value to false if you have not enough memory</td>
+</tr>
+</tbody>
+</table>
+<p>For example, if there are 10 million records ,and i have only 16 cores ,64GB memory, will be loaded to CarbonData table.
+Using the default configuration always fail in sort step. Modify carbon.properties as suggested below</p>
+<pre><code>carbon.number.of.cores.block.sort=1
+carbon.merge.sort.reader.thread=1
+carbon.sort.size=5000
+carbon.sort.file.write.buffer.size=5000
+carbon.merge.sort.prefetch=false
+</code></pre>
+<h2>
<a id="configurations-for-optimizing-carbondata-performance" class="anchor" href="#configurations-for-optimizing-carbondata-performance" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Configurations for Optimizing CarbonData Performance</h2>
<p>Recently we did some performance POC on CarbonData for Finance and telecommunication Field. It involved detailed queries and aggregation
scenarios. After the completion of POC, some of the configurations impacting the performance have been identified and tabulated below :</p>
@@ -396,6 +452,8 @@ scenarios. After the completion of POC, some of the configurations impacting the
</tr>
</tbody>
</table>
+<p>Note: If your CarbonData instance is provided only for query, you may specify the conf 'spark.speculation=true' which is conf
+in spark.</p>
</div>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/src/main/webapp/configuration-parameters.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/configuration-parameters.html b/src/main/webapp/configuration-parameters.html
index 3b2d562..63943eb 100644
--- a/src/main/webapp/configuration-parameters.html
+++ b/src/main/webapp/configuration-parameters.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/src/main/webapp/data-management.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/data-management.html b/src/main/webapp/data-management.html
index 0ae1ef8..b3a9c99 100644
--- a/src/main/webapp/data-management.html
+++ b/src/main/webapp/data-management.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
@@ -201,12 +201,12 @@ CarbonData store metadata. The table below lists the possible load status.</p>
</tr>
</tbody>
</table>
-<p>In case of failure, the error will be logged in error log. Details of loads can be seen with <a href="dml-operation-on-carbondata.html">SHOW SEGMENTS</a> command. The show segment command output consists of :</p>
+<p>In case of failure, the error will be logged in error log. Details of loads can be seen with <a href="dml-operation-on-carbondata.html#show-segments">SHOW SEGMENTS</a> command. The show segment command output consists of :</p>
<ul>
-<li>SegmentSequenceID</li>
-<li>START_TIME OF LOAD</li>
-<li>END_TIME OF LOAD</li>
-<li>LOAD STATUS</li>
+<li>SegmentSequenceId</li>
+<li>Status</li>
+<li>Load Start Time</li>
+<li>Load End Time</li>
</ul>
<p>The latest load will be displayed first in the output.</p>
<p>Refer to <a href="dml-operation-on-carbondata.html">DML operations on CarbonData</a> for load commands.</p>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/src/main/webapp/ddl-operation-on-carbondata.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/ddl-operation-on-carbondata.html b/src/main/webapp/ddl-operation-on-carbondata.html
index a12d15c..81e8664 100644
--- a/src/main/webapp/ddl-operation-on-carbondata.html
+++ b/src/main/webapp/ddl-operation-on-carbondata.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
@@ -165,10 +165,18 @@
<ul>
<li><a href="#create-table">CREATE TABLE</a></li>
<li><a href="#show-table">SHOW TABLE</a></li>
+<li>
+<a href="#alter-table">ALTER TABLE</a>
+<ul>
+<li><a href="#rename-table">RENAME TABLE</a></li>
+<li><a href="#add-column">ADD COLUMN</a></li>
+<li><a href="#drop-columns">DROP COLUMNS</a></li>
+<li><a href="#change-data-type">CHANGE DATA TYPE</a></li>
+</ul>
+</li>
<li><a href="#drop-table">DROP TABLE</a></li>
<li><a href="#compaction">COMPACTION</a></li>
<li><a href="#bucketing">BUCKETING</a></li>
-<li><a href="#table-rename">TABLE RENAME</a></li>
</ul>
<h2>
<a id="create-table" class="anchor" href="#create-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CREATE TABLE</h2>
@@ -236,8 +244,8 @@
<p>Column groups with more than one column are stored in row format, instead of columnar format. By default, each column is a separate column group.</p>
</li>
</ul>
-<pre><code>TBLPROPERTIES ('COLUMN_GROUPS'='(column1, column3),
-(Column4,Column5,Column6)')
+<pre><code>TBLPROPERTIES ('COLUMN_GROUPS'='(column1, column2),
+(Column3,Column4,Column5)')
</code></pre>
<ul>
<li>
@@ -313,12 +321,239 @@ By default inverted index is enabled. The user can disable the inverted index cr
<pre><code> SHOW TABLES IN ProductSchema;
</code></pre>
<h2>
+<a id="alter-table" class="anchor" href="#alter-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>ALTER TABLE</h2>
+<p>The following section shall discuss the commands to modify the physical or logical state of the existing table(s).</p>
+<h3>
+<a id="rename-table" class="anchor" href="#rename-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a><strong>RENAME TABLE</strong>
+</h3>
+<p>This command is used to rename the existing table.</p>
+<pre><code> ALTER TABLE [db_name.]table_name RENAME TO new_table_name;
+</code></pre>
+<h4>
+<a id="parameter-description-2" class="anchor" href="#parameter-description-2" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h4>
+<table>
+<thead>
+<tr>
+<th>Parameter</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>db_Name</td>
+<td>Name of the database. If this parameter is left unspecified, the current database is selected.</td>
+</tr>
+<tr>
+<td>table_name</td>
+<td>Name of the existing table.</td>
+</tr>
+<tr>
+<td>new_table_name</td>
+<td>New table name for the existing table.</td>
+</tr>
+</tbody>
+</table>
+<h4>
+<a id="usage-guidelines-1" class="anchor" href="#usage-guidelines-1" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h4>
+<ul>
+<li>
+<p>Queries that require the formation of path using the table name for reading carbon store files, running in parallel with Rename command might fail during the renaming operation.</p>
+</li>
+<li>
+<p>Renaming of Secondary index table(s) is not permitted.</p>
+</li>
+</ul>
+<h4>
+<a id="examples" class="anchor" href="#examples" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Examples:</h4>
+<pre><code> ALTER TABLE carbon RENAME TO carbondata;
+</code></pre>
+<pre><code> ALTER TABLE test_db.carbon RENAME TO test_db.carbondata;
+</code></pre>
+<h3>
+<a id="add-column" class="anchor" href="#add-column" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a><strong>ADD COLUMN</strong>
+</h3>
+<p>This command is used to add a new column to the existing table.</p>
+<pre><code> ALTER TABLE [db_name.]table_name ADD COLUMNS (col_name data_type,...)
+ TBLPROPERTIES('DICTIONARY_INCLUDE'='col_name,...',
+ 'DICTIONARY_EXCLUDE'='col_name,...',
+ 'DEFAULT.VALUE.COLUMN_NAME'='default_value');
+</code></pre>
+<h4>
+<a id="parameter-description-3" class="anchor" href="#parameter-description-3" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h4>
+<table>
+<thead>
+<tr>
+<th>Parameter</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>db_Name</td>
+<td>Name of the database. If this parameter is left unspecified, the current database is selected.</td>
+</tr>
+<tr>
+<td>table_name</td>
+<td>Name of the existing table.</td>
+</tr>
+<tr>
+<td>col_name data_type</td>
+<td>Name of comma-separated column with data type. Column names contain letters, digits, and underscores (_).</td>
+</tr>
+</tbody>
+</table>
+<p>NOTE: Do not name the column after name, tupleId, PositionId, and PositionReference when creating Carbon tables because they are used internally by UPDATE, DELETE, and secondary index.</p>
+<h4>
+<a id="usage-guidelines-2" class="anchor" href="#usage-guidelines-2" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h4>
+<ul>
+<li>
+<p>Apart from DICTIONARY_INCLUDE, DICTIONARY_EXCLUDE and default_value no other property will be read. If any other property name is specified, error will not be thrown, it will be ignored.</p>
+</li>
+<li>
+<p>If default value is not specified, then NULL will be considered as the default value for the column.</p>
+</li>
+<li>
+<p>For addition of column, if DICTIONARY_INCLUDE and DICTIONARY_EXCLUDE are not specified, then the decision will be taken based on data type of the column.</p>
+</li>
+</ul>
+<h4>
+<a id="examples-1" class="anchor" href="#examples-1" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Examples:</h4>
+<pre><code> ALTER TABLE carbon ADD COLUMNS (a1 INT, b1 STRING);
+</code></pre>
+<pre><code> ALTER TABLE carbon ADD COLUMNS (a1 INT, b1 STRING) TBLPROPERTIES('DICTIONARY_EXCLUDE'='b1');
+</code></pre>
+<pre><code> ALTER TABLE carbon ADD COLUMNS (a1 INT, b1 STRING) TBLPROPERTIES('DICTIONARY_INCLUDE'='a1');
+</code></pre>
+<pre><code> ALTER TABLE carbon ADD COLUMNS (a1 INT, b1 STRING) TBLPROPERTIES('DEFAULT.VALUE.a1'='10');
+</code></pre>
+<h3>
+<a id="drop-columns" class="anchor" href="#drop-columns" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a><strong>DROP COLUMNS</strong>
+</h3>
+<p>This command is used to delete a existing column or multiple columns in a table.</p>
+<pre><code> ALTER TABLE [db_name.]table_name DROP COLUMNS (col_name, ...);
+</code></pre>
+<h4>
+<a id="parameter-description-4" class="anchor" href="#parameter-description-4" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h4>
+<table>
+<thead>
+<tr>
+<th>Parameter</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>db_Name</td>
+<td>Name of the database. If this parameter is left unspecified, the current database is selected.</td>
+</tr>
+<tr>
+<td>table_name</td>
+<td>Name of the existing table.</td>
+</tr>
+<tr>
+<td>col_name</td>
+<td>Name of comma-separated column with data type. Column names contain letters, digits, and underscores (_)</td>
+</tr>
+</tbody>
+</table>
+<h4>
+<a id="usage-guidelines-3" class="anchor" href="#usage-guidelines-3" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h4>
+<ul>
+<li>
+<p>Deleting a column will also clear the dictionary files, provided the column is of type dictionary.</p>
+</li>
+<li>
+<p>For delete column operation, there should be at least one key column that exists in the schema after deletion else error message will be displayed and the operation shall fail.</p>
+</li>
+</ul>
+<h4>
+<a id="examples-2" class="anchor" href="#examples-2" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Examples:</h4>
+<p>If the table contains 4 columns namely a1, b1, c1, and d1.</p>
+<ul>
+<li><strong>To delete a single column:</strong></li>
+</ul>
+<pre><code> ALTER TABLE carbon DROP COLUMNS (b1);
+</code></pre>
+<pre><code> ALTER TABLE test_db.carbon DROP COLUMNS (b1);
+</code></pre>
+<ul>
+<li><strong>To delete multiple columns:</strong></li>
+</ul>
+<pre><code> ALTER TABLE carbon DROP COLUMNS (b1,c1);
+</code></pre>
+<pre><code> ALTER TABLE carbon DROP COLUMNS (b1,c1);
+</code></pre>
+<h3>
+<a id="change-data-type" class="anchor" href="#change-data-type" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a><strong>CHANGE DATA TYPE</strong>
+</h3>
+<p>This command is used to change the data type from INT to BIGINT or decimal precision from lower to higher.</p>
+<pre><code> ALTER TABLE [db_name.]table_name CHANGE col_name col_name changed_column_type;
+</code></pre>
+<h4>
+<a id="parameter-description-5" class="anchor" href="#parameter-description-5" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h4>
+<table>
+<thead>
+<tr>
+<th>Parameter</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>db_Name</td>
+<td>Name of the database. If this parameter is left unspecified, the current database is selected.</td>
+</tr>
+<tr>
+<td>table_name</td>
+<td>Name of the existing table.</td>
+</tr>
+<tr>
+<td>col_name</td>
+<td>Name of comma-separated column with data type. Column names contain letters, digits, and underscores (_).</td>
+</tr>
+<tr>
+<td>changed_column_type</td>
+<td>The change in the data type.</td>
+</tr>
+</tbody>
+</table>
+<h4>
+<a id="usage-guidelines-4" class="anchor" href="#usage-guidelines-4" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h4>
+<ul>
+<li>Change of decimal data type from lower precision to higher precision will only be supported for cases where there is no data loss.</li>
+</ul>
+<h4>
+<a id="valid-scenarios" class="anchor" href="#valid-scenarios" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Valid Scenarios</h4>
+<ul>
+<li>
+<p>Invalid scenario - Change of decimal precision from (10,2) to (10,5) is not valid as in this case only scale is increased but total number of digits remain the same.</p>
+</li>
+<li>
+<p>Valid scenario - Change of decimal precision from (10,2) to (12,3) is valid as the total number of digits are increased by 2 but scale is increased only by 1 which will not lead to any data loss.</p>
+</li>
+<li>
+<p>Note :The allowed range is 38,38 (precision, scale) and is a valid upper case scenario which is not resulting in data loss.</p>
+</li>
+</ul>
+<h4>
+<a id="examples-3" class="anchor" href="#examples-3" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Examples:</h4>
+<ul>
+<li><strong>Changing data type of column a1 from INT to BIGINT</strong></li>
+</ul>
+<pre><code> ALTER TABLE test_db.carbon CHANGE a1 a1 BIGINT;
+</code></pre>
+<ul>
+<li><strong>Changing decimal precision of column a1 from 10 to 18.</strong></li>
+</ul>
+<pre><code> ALTER TABLE test_db.carbon CHANGE a1 a1 DECIMAL(18,2);
+</code></pre>
+<h2>
<a id="drop-table" class="anchor" href="#drop-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>DROP TABLE</h2>
<p>This command is used to delete an existing table.</p>
<pre><code> DROP TABLE [IF EXISTS] [db_name.]table_name;
</code></pre>
<h3>
-<a id="parameter-description-2" class="anchor" href="#parameter-description-2" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h3>
+<a id="parameter-description-6" class="anchor" href="#parameter-description-6" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h3>
<table>
<thead>
<tr>
@@ -351,7 +586,7 @@ By default inverted index is enabled. The user can disable the inverted index cr
</code></pre>
<p>To get details about Compaction refer to <a href="data-management.html">Data Management</a></p>
<h3>
-<a id="parameter-description-3" class="anchor" href="#parameter-description-3" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h3>
+<a id="parameter-description-7" class="anchor" href="#parameter-description-7" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h3>
<table>
<thead>
<tr>
@@ -397,8 +632,8 @@ of columns is used.</p>
TBLPROPERTIES('BUCKETNUMBER'='noOfBuckets',
'BUCKETCOLUMNS'='columnname')
</code></pre>
-<h2>
-<a id="parameter-description-4" class="anchor" href="#parameter-description-4" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h2>
+<h3>
+<a id="parameter-description-8" class="anchor" href="#parameter-description-8" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h3>
<table>
<thead>
<tr>
@@ -420,8 +655,8 @@ of columns is used.</p>
</tr>
</tbody>
</table>
-<h2>
-<a id="usage-guidelines-1" class="anchor" href="#usage-guidelines-1" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h2>
+<h3>
+<a id="usage-guidelines-5" class="anchor" href="#usage-guidelines-5" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h3>
<ul>
<li>
<p>The feature is supported for Spark 1.6.2 onwards, but the performance optimization is evident from Spark 2.1 onwards.</p>
@@ -433,69 +668,25 @@ of columns is used.</p>
<p>Columns in the BUCKETCOLUMN parameter must be only dimension. The BUCKETCOLUMN parameter can not be a measure or a combination of measures and dimensions.</p>
</li>
</ul>
-<h2>
-<a id="example-" class="anchor" href="#example-" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Example :</h2>
+<h3>
+<a id="example-" class="anchor" href="#example-" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Example :</h3>
<pre><code> CREATE TABLE IF NOT EXISTS productSchema.productSalesTable (
productNumber Int,
+ saleQuantity Int,
productName String,
storeCity String,
storeProvince String,
productCategory String,
productBatch String,
- saleQuantity Int,
revenue Int)
STORED BY 'carbondata'
- TBLPROPERTIES ('COLUMN_GROUPS'='(productName,productNumber)',
+ TBLPROPERTIES ('COLUMN_GROUPS'='(productNumber,saleQuantity)',
'DICTIONARY_EXCLUDE'='productName',
- 'DICTIONARY_INCLUDE'='productNumber',
+ 'DICTIONARY_INCLUDE'='productNumber,saleQuantity',
'NO_INVERTED_INDEX'='productBatch',
'BUCKETNUMBER'='4',
'BUCKETCOLUMNS'='productName')
</code></pre>
-<h2>
-<a id="table-rename" class="anchor" href="#table-rename" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>TABLE RENAME</h2>
-<p>This command is used to rename the existing table.</p>
-<h3>
-<a id="syntax-1" class="anchor" href="#syntax-1" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Syntax</h3>
-<pre><code> ALTER TABLE [db_name.]table_name RENAME TO new_table_name;
-</code></pre>
-<h3>
-<a id="parameter-description-5" class="anchor" href="#parameter-description-5" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Parameter Description</h3>
-<table>
-<thead>
-<tr>
-<th>Parameter</th>
-<th>Description</th>
-</tr>
-</thead>
-<tbody>
-<tr>
-<td>db_name</td>
-<td>Name of the database. If this parameter is left unspecified, the current database is selected.</td>
-</tr>
-<tr>
-<td>table_name</td>
-<td>Name of the existing table.</td>
-</tr>
-<tr>
-<td>new_table_name</td>
-<td>New table name for the existing table.</td>
-</tr>
-</tbody>
-</table>
-<h3>
-<a id="usage-guidelines-2" class="anchor" href="#usage-guidelines-2" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h3>
-<p>Following conditions must be met for successful rename operation:</p>
-<ul>
-<li>Queries running in parallel which requires the formation of path using the table name for reading carbon store files might fail during this operation.</li>
-<li>Secondary index table rename is not permitted.</li>
-</ul>
-<h3>
-<a id="example-3" class="anchor" href="#example-3" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Example:</h3>
-<pre><code> ALTER TABLE carbon RENAME TO carbondata;
-
- ALTER TABLE test_db.carbon RENAME TO test_db.carbondata;
-</code></pre>
</div>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/src/main/webapp/dml-operation-on-carbondata.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/dml-operation-on-carbondata.html b/src/main/webapp/dml-operation-on-carbondata.html
index 52a70db..e0c234e 100644
--- a/src/main/webapp/dml-operation-on-carbondata.html
+++ b/src/main/webapp/dml-operation-on-carbondata.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/src/main/webapp/faq.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/faq.html b/src/main/webapp/faq.html
index 8567346..54fcc09 100644
--- a/src/main/webapp/faq.html
+++ b/src/main/webapp/faq.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
@@ -166,6 +166,7 @@
<li><a href="#how-to-specify-store-location-while-creating-carbon-session">How to specify store location while creating carbon session?</a></li>
<li><a href="#what-is-carbon-lock-type">What is Carbon Lock Type?</a></li>
<li><a href="#how-to-resolve-abstract-method-error">How to resolve Abstract Method Error?</a></li>
+<li><a href="#how-carbon-will-behave-when-execute-insert-operation-in-abnormal-scenarios">How Carbon will behave when execute insert operation in abnormal scenarios?</a></li>
</ul>
<h2>
<a id="what-are-bad-records" class="anchor" href="#what-are-bad-records" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>What are Bad Records?</h2>
@@ -214,6 +215,48 @@ The property carbon.lock.type configuration specifies the type of lock to be acq
<h2>
<a id="how-to-resolve-abstract-method-error" class="anchor" href="#how-to-resolve-abstract-method-error" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>How to resolve Abstract Method Error?</h2>
<p>In order to build CarbonData project it is necessary to specify the spark profile. The spark profile sets the Spark Version. You need to specify the <code>spark version</code> while using Maven to build project.</p>
+<h2>
+<a id="how-carbon-will-behave-when-execute-insert-operation-in-abnormal-scenarios" class="anchor" href="#how-carbon-will-behave-when-execute-insert-operation-in-abnormal-scenarios" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>How Carbon will behave when execute insert operation in abnormal scenarios?</h2>
+<p>Carbon support insert operation, you can refer to the syntax mentioned in <a href="http://carbondata.apache.org/dml-operation-on-carbondata" target=_blank>DML Operations on CarbonData</a>.
+First, create a soucre table in spark-sql and load data into this created table.</p>
+<pre><code>CREATE TABLE source_table(
+id String,
+name String,
+city String)
+ROW FORMAT DELIMITED FIELDS TERMINATED BY ",";
+</code></pre>
+<pre><code>SELECT * FROM source_table;
+id name city
+1 jack beijing
+2 erlu hangzhou
+3 davi shenzhen
+</code></pre>
+<p><strong>Scenario 1</strong> :</p>
+<p>Suppose, the column order in carbon table is different from source table, use script "SELECT * FROM carbon table" to query, will get the column order similar as source table, rather than in carbon table's column order as expected.</p>
+<pre><code>CREATE TABLE IF NOT EXISTS carbon_table(
+id String,
+city String,
+name String)
+STORED BY 'carbondata';
+</code></pre>
+<pre><code>INSERT INTO TABLE carbon_table SELECT * FROM source_table;
+</code></pre>
+<pre><code>SELECT * FROM carbon_table;
+id city name
+1 jack beijing
+2 erlu hangzhou
+3 davi shenzhen
+</code></pre>
+<p>As result shows, the second column is city in carbon table, but what inside is name, such as jack. This phenomenon is same with insert data into hive table.</p>
+<p>If you want to insert data into corresponding column in carbon table, you have to specify the column order same in insert statment.</p>
+<pre><code>INSERT INTO TABLE carbon_table SELECT id, city, name FROM source_table;
+</code></pre>
+<p><strong>Scenario 2</strong> :</p>
+<p>Insert operation will be failed when the number of column in carbon table is different from the column specified in select statement. The following insert operation will be failed.</p>
+<pre><code>INSERT INTO TABLE carbon_table SELECT id, city FROM source_table;
+</code></pre>
+<p><strong>Scenario 3</strong> :</p>
+<p>When the column type in carbon table is different from from the column specified in select statement. The insert operation will still success, but you may get NULL in result, because NULL will be substitute value when conversion type failed.</p>
</div>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/src/main/webapp/file-structure-of-carbondata.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/file-structure-of-carbondata.html b/src/main/webapp/file-structure-of-carbondata.html
index 564486b..985b116 100644
--- a/src/main/webapp/file-structure-of-carbondata.html
+++ b/src/main/webapp/file-structure-of-carbondata.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/b2222701/src/main/webapp/installation-guide.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/installation-guide.html b/src/main/webapp/installation-guide.html
index 717ca62..fd31349 100644
--- a/src/main/webapp/installation-guide.html
+++ b/src/main/webapp/installation-guide.html
@@ -67,7 +67,7 @@
target="_blank">Release Archive</a></li>
</ul>
</li>
- <li><a href="mainpage.html" class="">Documentation</a></li>
+ <li><a href="mainpage.html" class="active">Documentation</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Community <span class="caret"></span></a>
@@ -125,7 +125,7 @@
<tr>
<td style="width:80%">
<input type="text" name="q" size=" 5" maxlength="255" value=""
- class="search-input"/>
+ class="search-input" placeholder="Search...." required/>
</td>
<td style="width:20%">
<input type="submit" value="Search"/></td>
@@ -133,7 +133,7 @@
<tr>
<td align="left" style="font-size:75%" colspan="2">
<input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
- Only search for CarbonData
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
</td>
</tr>
</table>
@@ -324,7 +324,7 @@ mv carbondata.tar.gz carbonlib/
<tr>
<td>spark.driver.extraClassPath</td>
<td>Extra classpath entries to prepend to the classpath of the driver. <strong>NOTE</strong>: If SPARK_CLASSPATH is defined in spark-env.sh, then comment it and append the value in below parameter spark.driver.extraClassPath.</td>
-<td><code>$SPARK_HOME/carbonlib/carbonlib/*</code></td>
+<td><code>$SPARK_HOME/carbonlib/*</code></td>
</tr>
<tr>
<td>spark.driver.extraJavaOptions</td>