You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@carbondata.apache.org by ch...@apache.org on 2017/02/15 18:56:54 UTC
[1/4] incubator-carbondata-site git commit: updated the FAQs and
added Security page and Search Functionality
Repository: incubator-carbondata-site
Updated Branches:
refs/heads/asf-site b8a05236e -> b0c921fae
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/site/markdown/troubleshooting.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/troubleshooting.md b/src/site/markdown/troubleshooting.md
index b1bc9fa..2c892ae 100644
--- a/src/site/markdown/troubleshooting.md
+++ b/src/site/markdown/troubleshooting.md
@@ -21,6 +21,19 @@
This tutorial is designed to provide troubleshooting for end users and developers
who are building, deploying, and using CarbonData.
+* [Failed to load thrift libraries](#failed-to-load-thrift-libraries)
+* [Failed to launch the Spark Shell](#failed-to-launch-the-spark-shell)
+* [Query Failure with Generic Error on the Beeline](#query-failure-with-generic-error-on-the-beeline)
+* [Failed to execute load query on cluster](#failed-to-execute-load-query-on-cluster)
+* [Failed to execute insert query on cluster](#failed-to-execute-insert-query-on-cluster)
+* [Failed to connect to hiveuser with thrift](#failed-to-connect-to-hiveuser-with-thrift)
+* [Failure to read the metastore db during table creation](#failure-to-read-the-metastore-db-during-table-creation)
+* [Failed to load data on the cluster](#failed-to-load-data-on-the-cluster)
+* [Failed to insert data on the cluster](#failed-to-insert-data-on-the-cluster)
+* [Failed to execute Concurrent Operations](#failed-to-execute-concurrent-operations)
+* [Failed to create a table with a single numeric column](#failed-to-create-a-table-with-a-single-numeric-column)
+* [Data Failure because of Bad Records](#data-failure-because-of-bad-records)
+
## Failed to load thrift libraries
**Symptom**
@@ -86,6 +99,41 @@ who are building, deploying, and using CarbonData.
```
Note : Refrain from using "mvn clean package" without specifying the profile.
+
+## Query Failure with Generic Error on the Beeline
+
+ **Symptom**
+
+ Query fails on the executor side and generic error message is printed on the beeline console
+
+ 
+
+ **Possible Causes**
+
+ - In Query flow, Table B-Tree will be loaded into memory on the driver side and filter condition is validated against the min-max of each block to identify false positive,
+ Once the blocks are selected, based on number of available executors, blocks will be distributed to each executor node as shown in below driver logs snapshot
+
+ 
+
+ - When the error occurs in driver side while b-tree loading or block distribution, detail error message will be printed on the beeline console and error trace will be printed on the driver logs.
+
+ - When the error occurs in the executor side, generic error message will be printed as shown in issue description.
+
+ 
+
+ - Details of the failed stages can be seen in the Spark Application UI by clicking on the failed stages on the failed job as shown in previous snapshot
+
+ 
+
+ **Procedure**
+
+ Details of the error can be analyzed in details using executor logs available in stdout
+
+ 
+
+ Below snapshot shows executor logs with error message for query failure which can be helpful to locate the error
+
+ 
## Failed to execute load query on cluster.
@@ -229,11 +277,11 @@ who are building, deploying, and using CarbonData.
2. For the changes to take effect, restart the Spark cluster.
-## Failed to execute Concurrent Operations(Load,Insert,Update) on table by multiple workers.
+## Failed to execute Concurrent Operations.
**Symptom**
- Execution fails with the following exception :
+ Execution of Concurrent Operations (Load,Insert,Update) on table by multiple workers fails with the following exception :
```
Table is locked for updation.
@@ -264,3 +312,29 @@ who are building, deploying, and using CarbonData.
**Procedure**
A single column that can be considered as dimension is mandatory for table creation.
+
+## Data Failure because of Bad Records
+
+ **Symptom**
+
+ Data Loading fails with the following exception
+
+ ```
+ Error: java.lang.Exception: Data load failed due to Bad record
+ ```
+
+ **Possible Causes**
+
+ The parameter BAD_RECORDS_ACTION has not been specified in the Query.
+
+ **Procedure**
+
+ Set the following parameter in the load command OPTIONS as shown below :
+
+ 'BAD_RECORDS_ACTION'='FORCE\u2018
+
+ *Example :*
+
+ ```
+ LOAD DATA INPATH 'hdfs://hacluster/user/loader/moredata01.csv' INTO TABLE flow_carbon_256b OPTIONS('DELIMITER'=',', 'BAD_RECORDS_ACTION'='FORCE');
+ ```
[4/4] incubator-carbondata-site git commit: Added Search,
Security and Faqs, and fix some issues
Posted by ch...@apache.org.
Added Search,Security and Faqs,and fix some issues
Project: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/commit/b0c921fa
Tree: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/tree/b0c921fa
Diff: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/diff/b0c921fa
Branch: refs/heads/asf-site
Commit: b0c921faed7e2cae993d9e78f433b9b55c61e87b
Parents: b8a0523 7655ecd
Author: chenliang613 <ch...@huawei.com>
Authored: Wed Feb 15 10:55:51 2017 -0800
Committer: chenliang613 <ch...@huawei.com>
Committed: Wed Feb 15 10:55:51 2017 -0800
----------------------------------------------------------------------
content/css/style.css | 45 ++-
content/docs/latest/faq.html | 193 +++++++++-
.../latest/file-structure-of-carbondata.html | 5 +-
.../latest/images/query_failure_beeline.png | Bin 0 -> 48077 bytes
.../docs/latest/images/query_failure_issue.png | Bin 0 -> 316072 bytes
.../latest/images/query_failure_job_details.png | Bin 0 -> 113176 bytes
.../docs/latest/images/query_failure_logs.png | Bin 0 -> 441678 bytes
.../latest/images/query_failure_procedure.png | Bin 0 -> 80817 bytes
.../latest/images/query_failure_spark_ui.png | Bin 0 -> 212907 bytes
content/docs/latest/mainpage.html | 25 +-
content/docs/latest/troubleshooting.html | 373 ++++++++++++++-----
content/index.html | 34 +-
content/js/custom.js | 9 +-
content/meetup.html | 358 +++++++++++-------
content/pdf/maven-pdf-plugin.pdf | Bin 156769 -> 162057 bytes
content/security.html | 173 +++++++++
src/main/webapp/css/style.css | 45 ++-
src/main/webapp/docs/latest/faq.html | 193 +++++++++-
.../latest/images/query_failure_beeline.png | Bin 0 -> 48077 bytes
.../docs/latest/images/query_failure_issue.png | Bin 0 -> 316072 bytes
.../latest/images/query_failure_job_details.png | Bin 0 -> 113176 bytes
.../docs/latest/images/query_failure_logs.png | Bin 0 -> 441678 bytes
.../latest/images/query_failure_procedure.png | Bin 0 -> 80817 bytes
.../latest/images/query_failure_spark_ui.png | Bin 0 -> 212907 bytes
src/main/webapp/docs/latest/mainpage.html | 25 +-
.../webapp/docs/latest/troubleshooting.html | 373 ++++++++++++++-----
src/main/webapp/index.html | 30 +-
src/main/webapp/js/custom.js | 9 +-
src/main/webapp/meetup.html | 358 +++++++++++-------
src/main/webapp/security.html | 173 +++++++++
src/site/markdown/faq.md | 114 ++++--
.../markdown/images/query_failure_beeline.png | Bin 0 -> 48077 bytes
.../markdown/images/query_failure_issue.png | Bin 0 -> 316072 bytes
.../images/query_failure_job_details.png | Bin 0 -> 113176 bytes
src/site/markdown/images/query_failure_logs.png | Bin 0 -> 441678 bytes
.../markdown/images/query_failure_procedure.png | Bin 0 -> 80817 bytes
.../markdown/images/query_failure_spark_ui.png | Bin 0 -> 212907 bytes
src/site/markdown/troubleshooting.md | 78 +++-
38 files changed, 2054 insertions(+), 559 deletions(-)
----------------------------------------------------------------------
[3/4] incubator-carbondata-site git commit: updated the FAQs and
added Security page and Search Functionality
Posted by ch...@apache.org.
updated the FAQs and added Security page and Search Functionality
Project: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/commit/7655ecd6
Tree: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/tree/7655ecd6
Diff: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/diff/7655ecd6
Branch: refs/heads/asf-site
Commit: 7655ecd691a898a168620139558fe3e04de67c70
Parents: b8a0523
Author: PallaviSingh1992 <pa...@yahoo.co.in>
Authored: Tue Feb 14 18:14:10 2017 +0530
Committer: PallaviSingh1992 <pa...@yahoo.co.in>
Committed: Wed Feb 15 16:26:46 2017 +0530
----------------------------------------------------------------------
content/css/style.css | 45 ++-
content/docs/latest/faq.html | 193 +++++++++-
.../latest/file-structure-of-carbondata.html | 5 +-
.../latest/images/query_failure_beeline.png | Bin 0 -> 48077 bytes
.../docs/latest/images/query_failure_issue.png | Bin 0 -> 316072 bytes
.../latest/images/query_failure_job_details.png | Bin 0 -> 113176 bytes
.../docs/latest/images/query_failure_logs.png | Bin 0 -> 441678 bytes
.../latest/images/query_failure_procedure.png | Bin 0 -> 80817 bytes
.../latest/images/query_failure_spark_ui.png | Bin 0 -> 212907 bytes
content/docs/latest/mainpage.html | 25 +-
content/docs/latest/troubleshooting.html | 373 ++++++++++++++-----
content/index.html | 34 +-
content/js/custom.js | 9 +-
content/meetup.html | 358 +++++++++++-------
content/pdf/maven-pdf-plugin.pdf | Bin 156769 -> 162057 bytes
content/security.html | 173 +++++++++
src/main/webapp/css/style.css | 45 ++-
src/main/webapp/docs/latest/faq.html | 193 +++++++++-
.../latest/images/query_failure_beeline.png | Bin 0 -> 48077 bytes
.../docs/latest/images/query_failure_issue.png | Bin 0 -> 316072 bytes
.../latest/images/query_failure_job_details.png | Bin 0 -> 113176 bytes
.../docs/latest/images/query_failure_logs.png | Bin 0 -> 441678 bytes
.../latest/images/query_failure_procedure.png | Bin 0 -> 80817 bytes
.../latest/images/query_failure_spark_ui.png | Bin 0 -> 212907 bytes
src/main/webapp/docs/latest/mainpage.html | 25 +-
.../webapp/docs/latest/troubleshooting.html | 373 ++++++++++++++-----
src/main/webapp/index.html | 30 +-
src/main/webapp/js/custom.js | 9 +-
src/main/webapp/meetup.html | 358 +++++++++++-------
src/main/webapp/security.html | 173 +++++++++
src/site/markdown/faq.md | 114 ++++--
.../markdown/images/query_failure_beeline.png | Bin 0 -> 48077 bytes
.../markdown/images/query_failure_issue.png | Bin 0 -> 316072 bytes
.../images/query_failure_job_details.png | Bin 0 -> 113176 bytes
src/site/markdown/images/query_failure_logs.png | Bin 0 -> 441678 bytes
.../markdown/images/query_failure_procedure.png | Bin 0 -> 80817 bytes
.../markdown/images/query_failure_spark_ui.png | Bin 0 -> 212907 bytes
src/site/markdown/troubleshooting.md | 78 +++-
38 files changed, 2054 insertions(+), 559 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/css/style.css
----------------------------------------------------------------------
diff --git a/content/css/style.css b/content/css/style.css
index 3540959..2aa6248 100644
--- a/content/css/style.css
+++ b/content/css/style.css
@@ -840,15 +840,15 @@ p img{ max-width: 100% }
.meetup-info .meetup-location { font-size:16px; color: #444; margin-bottom: 10px;}
.meetup-info p { font-size:13px; color: #444; line-height: 18px; margin: 0 }
-.meetup-info i {font-size: 60px; color: #ff943d; border: 1px dashed #ff943d; padding: 15px; margin-bottom:6px; background-color: #fff;}
-.meetup-info:hover{background-color: #fff; border-color: #ff943d;}
+.meetup-info i {font-size: 60px; color: #ff943d; /*border: 1px dashed #ff943d;*/ padding: 15px; margin-bottom:6px; background-color: #fff;}
+.meetup-info:hover{background-color: #fff; border-color: #ff943d;}
.meetup-info .viewlink{ font-size:14px;}
.meetup-info .meetup-icon{ text-align: right}
.meetup-info .viewlink {font-size: 14px; width: 84px; background: #ff943d; display: inline-block; text-align: center;
padding: 4px 2px; color: #fff; font-weight: 600;}
.meetup-info .viewlink:hover {background: #dd6300;}
-.meetup-info i:hover {color: #dd6300; border: 1px dashed #dd6300; }
+.meetup-info i:hover {color: #dd6300; /*border: 1px dashed #dd6300; */}
.feedbackbox{width:80%; margin:20px auto; padding:20px; border:1px solid #eee; background-color: #fff;
box-shadow: 0 3px 2px #aaa }
@@ -902,6 +902,41 @@ box-shadow: 0 3px 2px #aaa }
height: 36px;
}
+#search-box{ position: relative; display: none;}
+.search-block{
+ border: 5px solid #ff943d;
+ padding: 6px;
+ width: 24em;
+ position: absolute;
+ background: #f8f8f8;
+ right: 0;
+
+}
+
+.search-block input[type="text"] {
+ width: 244px;
+ border: 1px solid #ff9443;
+ padding: 0 5px;
+}
+.search-block input[type="submit"] {
+ width: 70px;
+ color: white;
+ background: #ff943d;
+ border: 1px solid #ff943d;
+ font-size: 14px;
+
+}
+
+.search-block input:focus{ border: 1px solid #dd6300;}
+
+ .security-block {background: #fff; margin: 15px 0; padding:5px 15px;}
+ .security-block p{ margin:0 5px 20px;
+ font-size: 14px;
+ color: #444;
+ line-height: 20px;
+ display: block;
+ text-align: justify}
+
/* ------------------- End Documentation Style here*/
@@ -967,7 +1002,11 @@ box-shadow: 0 3px 2px #aaa }
}
+/* Media Close*/
+
+.mobile{ display: none;}
+.desktop{ display: block;}
/* Media Close*/
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/docs/latest/faq.html
----------------------------------------------------------------------
diff --git a/content/docs/latest/faq.html b/content/docs/latest/faq.html
index 3a7a354..d817823 100644
--- a/content/docs/latest/faq.html
+++ b/content/docs/latest/faq.html
@@ -15,13 +15,188 @@
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
---><h1>FAQs</h1>
+-->
+<h1 id="faqs">FAQs</h1>
<ul>
- <li><p><strong>Auto Compaction not Working</strong></p><p>The Property carbon.enable.auto.load.merge in carbon.properties need to be set to true.</p></li>
- <li><p><strong>Getting Abstract method error</strong></p><p>You need to specify the spark version while using Maven to build project.</p></li>
- <li><p><strong>Getting NotImplementedException for subquery using IN and EXISTS</strong></p><p>Subquery with in and exists not supported in CarbonData.</p></li>
- <li><p><strong>Getting Exceptions on creating a view</strong></p><p>View not supported in CarbonData.</p></li>
- <li><p><strong>How to verify if ColumnGroups have been created as desired.</strong></p><p>Try using desc table query.</p></li>
- <li><p><strong>Did anyone try to run CarbonData on windows? Is it supported on Windows?</strong></p><p>We may provide support for windows in future. You are welcome to contribute if you want to add the support :)</p></li>
- <li><p><strong>Can we execute Concurrent Operations(Load,Insert,Update) on table by multiple workers.</strong></p><p>Concurrency is not supported in current release of CarbonData.</p></li>
-</ul>
\ No newline at end of file
+ <li>
+ <p><a href="#can-we-preserve-segments-from-compaction">Can we preserve Segments from
+ Compaction?</a></p>
+ </li>
+ <li>
+ <p><a href="#can-we-disable-horizontal-compaction">Can we disable horizontal compaction?</a></p>
+ </li>
+ <li>
+ <p><a href="#what-is-horizontal-compaction">What is horizontal compaction?</a></p>
+ </li>
+ <li>
+ <p><a href="#how-to-enable-compaction-while-data-loading">How to enable Compaction while data
+ loading?</a></p>
+ </li>
+ <li>
+ <p><a href="#where-are-bad-records-stored-in-carbondata">Where are Bad Records Stored in
+ CarbonData?</a></p>
+ </li>
+ <li>
+ <p><a href="#what-are-bad-records">What are Bad Records?</a></p>
+ </li>
+ <li>
+ <p><a href="#can-we-use-carbondata-on-standalone-spark-cluster">Can we use CarbonData on
+ Standalone Spark Cluster?</a></p>
+ </li>
+ <li>
+ <p><a href="#what-versions-of-apache-spark-are-compatible-with-carbondata">What versions of
+ Apache Spark are Compatible with CarbonData?</a></p>
+ </li>
+ <li>
+ <p><a href="#can-we-load-data-from-excel">Can we Load Data from excel?</a></p>
+ </li>
+ <li>
+ <p><a href="#how-to-enable-single-pass-data-loading">How to enable Single Pass Data Loading?</a>
+ </p>
+ </li>
+ <li>
+ <p><a href="#what-is-single-pass-data-loading">What is Single Pass Data Loading?</a></p>
+ </li>
+ <li>
+ <p><a href="#how-to-specify-the-data-loading-format-for-carbondata">How to specify the data
+ loading format for CarbonData ?</a></p>
+ </li>
+ <li>
+ <p><a href="#how-to-resolve-store-location-can-not-be-found">How to resolve store location can\u2019t
+ be found?</a></p>
+ </li>
+ <li>
+ <p><a href="">What is carbon.lock.type?</a></p>
+ </li>
+ <li>
+ <p><a href="#how-to-enable-auto-compaction">How to enable Auto Compaction?</a></p>
+ </li>
+ <li>
+ <p><a href="#how-to-resolve-abstract-method-error">How to resolve Abstract Method Error?</a></p>
+ </li>
+ <li>
+ <p><a href="#getting-exception-on-creating-a-view">Getting Exception on Creating a View</a></p>
+ </li>
+ <li>
+ <p><a href="#is-carbondata-supported-for-windows">Is CarbonData supported for Windows?</a></p>
+ </li>
+
+</ul>
+
+<h2 id="can-we-preserve-segments-from-compaction">Can we preserve Segments from Compaction?</h2>
+<p>If you want to preserve number of segments from being compacted then you can set the property
+ <strong>carbon.numberof.preserve.segments</strong> equal to the <strong>value of number of
+ segments to be preserved</strong>.</p>
+<p>Note : <em>No segments are preserved by Default.</em></p>
+
+<h2 id="can-we-disable-horizontal-compaction">Can we disable horizontal compaction?</h2>
+<p>Yes, to disable horizontal compaction, set <strong>carbon.horizontal.compaction.enable</strong>
+ to <code>FALSE</code> in carbon.properties file.</p>
+
+<h2 id="what-is-horizontal-compaction">What is horizontal compaction?</h2>
+<p>Compaction performed after Update and Delete operations is referred as Horizontal Compaction.
+ After every DELETE and UPDATE operation, horizontal compaction may occur in case the delta
+ (DELETE/ UPDATE) files becomes more than specified threshold.</p>
+<p>By default the parameter <strong>carbon.horizontal.compaction.enable</strong> enabling the
+ horizontal compaction is set to <code>TRUE</code>.</p>
+
+<h2 id="how-to-enable-compaction-while-data-loading">How to enable Compaction while data
+ loading?</h2>
+<p>To enable compaction while data loading, set <strong>carbon.enable.auto.load.merge</strong> to
+ <code>TRUE</code> in carbon.properties file.</p>
+
+<h2 id="where-are-bad-records-stored-in-carbondata">Where are Bad Records Stored in CarbonData?</h2>
+<p>The bad records are stored at the location set in carbon.badRecords.location in carbon.properties
+ file.<br>
+ By default <strong>carbon.badRecords.location</strong> specifies the following location <code>/opt/Carbon/Spark/badrecords</code>.
+</p>
+
+<h2 id="what-are-bad-records">What are Bad Records?</h2>
+<p>Records that fail to get loaded into the CarbonData due to data type incompatibility are
+ classified as Bad Records.</p>
+
+<h2 id="can-we-use-carbondata-on-standalone-spark-cluster">Can we use CarbonData on Standalone Spark
+ Cluster?</h2>
+<p>Yes, CarbonData can be used on a Standalone spark cluster. But using a standalone cluster has
+ following limitations:</p>
+<ul>
+ <li>single node cluster cannot be scaled up</li>
+ <li>the maximum memory and the CPU computation power has a fixed limit</li>
+ <li>the number of processors are limited in a single node cluster</li>
+</ul>
+<p>To harness the actual speed of execution of CarbonData on petabytes of data, it is suggested to
+ use a Multinode Cluster.</p>
+
+<h2 id="what-versions-of-apache-spark-are-compatible-with-carbondata">What versions of Apache Spark
+ are Compatible with CarbonData?</h2>
+<p>Currently <strong>Spark 1.6.2</strong> and <strong>Spark 2.1</strong> is compatible with
+ CarbonData.</p>
+
+<h2 id="can-we-load-data-from-excel">Can we Load Data from excel?</h2>
+<p>Yes, the data can be loaded from excel provided the data is in CSV format.</p>
+
+<h2 id="how-to-enable-single-pass-data-loading">How to enable Single Pass Data Loading?</h2>
+<p>You need to set <strong>SINGLE_PASS</strong> to <code>True</code> and append it to
+ <code>OPTIONS</code> Section in the query as demonstrated in the Load Query below :</p>
+<pre><code>LOAD DATA local inpath '/opt/rawdata/data.csv' INTO table carbontable
+OPTIONS('DELIMITER'=',', 'QUOTECHAR'='"','FILEHEADER'='empno,empname,designation','USE_KETTLE'='FALSE')
+</code></pre>
+<p>Refer to <a
+ href="https://github.com/PallaviSingh1992/incubator-carbondata/blob/6b4dd5f3dea8c93839a94c2d2c80ab7a799cf209/docs/dml-operation-on-carbondata.md">DML-operations-in-CarbonData</a>
+ for more details and example.</p>
+
+<h2 id="what-is-single-pass-data-loading">What is Single Pass Data Loading?</h2>
+<p>Single Pass Loading enables single job to finish data loading with dictionary generation on the
+ fly. It enhances performance in the scenarios where the subsequent data loading after initial
+ load involves fewer incremental updates on the dictionary.<br>
+ This option specifies whether to use single pass for loading data or not. By default this option
+ is set to <code>FALSE</code>.</p>
+
+<h2 id="how-to-specify-the-data-loading-format-for-carbondata">How to specify the data loading
+ format for CarbonData?</h2>
+<p>Edit carbon.properties file. Modify the value of parameter
+ <strong>carbon.data.file.version</strong>.<br>
+ Setting the parameter <strong>carbon.data.file.version</strong> to <code>1</code> will support
+ data loading in <code>old format(0.x version)</code> and setting <strong>carbon.data.file.version</strong>
+ to <code>2</code> will support data loading in <code>new format(1.x onwards)</code> only.<br>
+ By default the data loading is supported using the new format.</p>
+
+<h2 id="how-to-resolve-store-location-can-not-be-found">How to resolve store location can not be
+ found?</h2>
+<p>Try creating <code>carbonsession</code> with <code>storepath</code> specified in the following
+ manner :</p>
+<pre><code>val carbon = SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession(<store_path>)
+</code></pre>
+<p>Example:</p>
+<pre><code>val carbon = SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession("hdfs://localhost:9000/carbon/store ")
+</code></pre>
+
+<h2 id="what-is-carbon-lockt-ype">What is carbon.lock.type?</h2>
+<p>This property configuration specifies the type of lock to be acquired during concurrent
+ operations on table. This property can be set with the following values :</p>
+<ul>
+ <li><strong>LOCALLOCK</strong> : This Lock is created on local file system as file. This lock is
+ useful when only one spark driver (thrift server) runs on a machine and no other CarbonData
+ spark application is launched concurrently.
+ </li>
+ <li><strong>HDFSLOCK</strong> : This Lock is created on HDFS file system as file. This lock is
+ useful when multiple CarbonData spark applications are launched and no ZooKeeper is running
+ on cluster and the HDFS supports, file based locking.
+ </li>
+</ul>
+
+<h2 id="how-to-enable-auto-compaction">How to enable Auto Compaction?</h2>
+<p>To enable compaction set <strong>carbon.enable.auto.load.merge</strong> to <code>TRUE</code> in
+ the carbon.properties file.</p>
+
+<h2 id="how-to-resolve-abstract-method-error">How to resolve Abstract Method Error?</h2>
+<p>You need to specify the <code>spark version</code> while using Maven to build project.</p>
+
+<h2 id="getting-exception-on-creating-a-view">Getting Exception on Creating a View</h2>
+<p>View not supported in CarbonData.</p>
+
+<h2 id="is-carbondata-supported-for-windows">Is CarbonData supported for Windows?</h2>
+<p>We may provide support for windows in future. You are welcome to contribute if you want to add
+ the support :)</p>
+
+</body></html>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/docs/latest/file-structure-of-carbondata.html
----------------------------------------------------------------------
diff --git a/content/docs/latest/file-structure-of-carbondata.html b/content/docs/latest/file-structure-of-carbondata.html
index 19401b9..3ca022b 100644
--- a/content/docs/latest/file-structure-of-carbondata.html
+++ b/content/docs/latest/file-structure-of-carbondata.html
@@ -1,6 +1,7 @@
-<h1>CarbonData File Structure</h1><p>CarbonData files contain groups of data called blocklets, along with all required information like schema, offsets and indices etc, in a file footer, co-located in HDFS.</p><p>The file footer can be read once to build the indices in memory, which can be utilized for optimizing the scans and processing for all subsequent queries.</p><p>Each blocklet in the file is further divided into chunks of data called data chunks. Each data chunk is organized either in columnar format or row format, and stores the data of either a single column or a set of columns. All blocklets in a file contain the same number and type of data chunks.</p><p><img src="../../../webapp/docs/latest/images/carbon_data_file_structure_new.png?raw=true" alt="CarbonData File Structure" /></p><p>Each data chunk contains multiple groups of data called as pages. There are three types of pages.</p>
+<h1>CarbonData File Structure</h1><p>CarbonData files contain groups of data called blocklets, along with all required information like schema, offsets and indices etc, in a file footer, co-located in HDFS.</p><p>The file footer can be read once to build the indices in memory, which can be utilized for optimizing the scans and processing for all subsequent queries.</p><p>Each blocklet in the file is further divided into chunks of data called data chunks. Each data chunk is organized either in columnar format or row format, and stores the data of either a single column or a set of columns. All blocklets in a file contain the same number and type of data chunks.</p><p><img src="../../docs/latest/images/carbon_data_file_structure_new.png?raw=true" alt="CarbonData File Structure" /></p><p>Each data chunk contains multiple groups of data called as pages. There are three types of pages.</p>
<ul>
<li>Data Page: Contains the encoded data of a column/group of columns.</li>
<li>Row ID Page (optional): Contains the row ID mappings used when the data page is stored as an inverted index.</li>
<li>RLE Page (optional): Contains additional metadata used when the data page is RLE coded.</li>
-</ul><p><img src="../../../webapp/docs/latest/images/carbon_data_format_new.png?raw=true" alt="CarbonData File Format" /></p>
\ No newline at end of file
+</ul><p><img src="../../docs/latest/images/carbon_data_format_new.png?raw=true" alt="CarbonData File Format" /></p>
+
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/docs/latest/images/query_failure_beeline.png
----------------------------------------------------------------------
diff --git a/content/docs/latest/images/query_failure_beeline.png b/content/docs/latest/images/query_failure_beeline.png
new file mode 100644
index 0000000..e4ec22b
Binary files /dev/null and b/content/docs/latest/images/query_failure_beeline.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/docs/latest/images/query_failure_issue.png
----------------------------------------------------------------------
diff --git a/content/docs/latest/images/query_failure_issue.png b/content/docs/latest/images/query_failure_issue.png
new file mode 100644
index 0000000..87270d2
Binary files /dev/null and b/content/docs/latest/images/query_failure_issue.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/docs/latest/images/query_failure_job_details.png
----------------------------------------------------------------------
diff --git a/content/docs/latest/images/query_failure_job_details.png b/content/docs/latest/images/query_failure_job_details.png
new file mode 100644
index 0000000..26e607d
Binary files /dev/null and b/content/docs/latest/images/query_failure_job_details.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/docs/latest/images/query_failure_logs.png
----------------------------------------------------------------------
diff --git a/content/docs/latest/images/query_failure_logs.png b/content/docs/latest/images/query_failure_logs.png
new file mode 100644
index 0000000..8fbdfa6
Binary files /dev/null and b/content/docs/latest/images/query_failure_logs.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/docs/latest/images/query_failure_procedure.png
----------------------------------------------------------------------
diff --git a/content/docs/latest/images/query_failure_procedure.png b/content/docs/latest/images/query_failure_procedure.png
new file mode 100644
index 0000000..9d2c81f
Binary files /dev/null and b/content/docs/latest/images/query_failure_procedure.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/docs/latest/images/query_failure_spark_ui.png
----------------------------------------------------------------------
diff --git a/content/docs/latest/images/query_failure_spark_ui.png b/content/docs/latest/images/query_failure_spark_ui.png
new file mode 100644
index 0000000..1802760
Binary files /dev/null and b/content/docs/latest/images/query_failure_spark_ui.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/docs/latest/mainpage.html
----------------------------------------------------------------------
diff --git a/content/docs/latest/mainpage.html b/content/docs/latest/mainpage.html
index b8d0346..7e62350 100644
--- a/content/docs/latest/mainpage.html
+++ b/content/docs/latest/mainpage.html
@@ -71,9 +71,11 @@
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Community <span class="caret"></span></a>
<ul class="dropdown-menu">
- <li><a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Contributing+to+CarbonData" target="_blank">Contributing to CarbonData</a></li>
+ <li><a href="https://github.com/apache/incubator-carbondata/blob/master/docs/How-to-contribute-to-Apache-CarbonData.md"
+ target="_blank">Contributing to CarbonData</a></li></li>
<li><a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Committers" target="_blank">Project Committers</a></li>
<li><a href="../../meetup.html">CarbonData Meetups </a></li>
+ <li><a href="../../security.html">Apache CarbonData Security</a></li>
</ul>
</li>
<li class="dropdown">
@@ -96,8 +98,29 @@
</ul>
</li>
+ <li>
+ <a href="#" id="search-icon" ><i class="fa fa-search" aria-hidden="true"></i></a>
+
+ </li>
+
</ul>
</div><!--/.nav-collapse -->
+ <div id="search-box" >
+ <form method="get" action="http://www.google.com/search">
+ <div class="search-block">
+ <table border="0" cellpadding="0" width="100%">
+ <tr>
+ <td style="width:80%">
+ <input type="text" name="q" size=" 5" maxlength="255" value="" class="search-input" />
+ </td>
+ <td style="width:20%">
+ <input type="submit" value="Search" /></td></tr>
+ <tr><td align="left" style="font-size:75%" colspan="2">
+ <input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked /> Only search for CarbonData </td></tr>
+ </table>
+ </div>
+ </form>
+ </div>
</div>
</nav>
</header> <!-- end Header part -->
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/docs/latest/troubleshooting.html
----------------------------------------------------------------------
diff --git a/content/docs/latest/troubleshooting.html b/content/docs/latest/troubleshooting.html
index a3fb1a5..7be27da 100644
--- a/content/docs/latest/troubleshooting.html
+++ b/content/docs/latest/troubleshooting.html
@@ -15,116 +15,283 @@
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
---><h1>Troubleshooting</h1><p>This tutorial is designed to provide troubleshooting for end users and
- developers who are building, deploying, and using CarbonData.</p><h2>Failed to load thrift
- libraries</h2><p><strong>Symptom</strong></p><p>Thrift throws following exception :</p><p><code>
- thrift: error while loading shared libraries:
- libthriftc.so.0: cannot open shared object file: No such file or directory
-</code></p><p><strong>Possible Cause</strong></p><p>The complete path to the directory containing
- the libraries is not configured correctly.</p><p><strong>Procedure</strong></p><p>Follow the
- steps below to ensure loading of libraries appropriately :</p>
+--><h1>Troubleshooting</h1>
+<p>This tutorial is designed to provide troubleshooting for end users and developers
+ who are building, deploying, and using CarbonData.</p>
+<ul>
+ <li><a href="#failed-to-load-thrift-libraries">Failed to load thrift libraries</a></li>
+ <li><a href="#failed-to-launch-the-spark-shell">Failed to launch the Spark Shell</a></li>
+ <li><a href="#query-failure-with-generic-error-on-the-beeline">Query Failure with Generic Error
+ on the Beeline</a></li>
+ <li><a href="#failed-to-execute-load-query-on-cluster">Failed to execute load query on
+ cluster</a></li>
+ <li><a href="#failed-to-execute-insert-query-on-cluster">Failed to execute insert query on
+ cluster</a></li>
+ <li><a href="#failed-to-connect-to-hiveuser-with-thrift">Failed to connect to hiveuser with
+ thrift</a></li>
+ <li><a href="#failure-to-read-the-metastore-db-during-table-creation">Failure to read the
+ metastore db during table creation</a></li>
+ <li><a href="#failed-to-load-data-on-the-cluster">Failed to load data on the cluster</a></li>
+ <li><a href="#failed-to-insert-data-on-the-cluster">Failed to insert data on the cluster</a>
+ </li>
+ <li><a href="#failed-to-execute-concurrent-operations">Failed to execute Concurrent
+ Operations</a></li>
+ <li><a href="#failed-to-create-a-table-with-a-single-numeric-column">Failed to create a table
+ with a single numeric column</a></li>
+ <li><a href="#data-failure-because-of-bad-records">Data Failure because of Bad Records</a></li>
+</ul>
+<h2 id="failed-to-load-thrift-libraries">Failed to load thrift libraries</h2>
+<p><strong>Symptom</strong></p>
+<p>Thrift throws following exception :</p>
+<pre><code>thrift: error while loading shared libraries:
+libthriftc.so.0: cannot open shared object file: No such file or directory
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The complete path to the directory containing the libraries is not configured correctly.</p>
+<p><strong>Procedure</strong></p>
+<p>Follow the steps below to ensure loading of libraries appropriately :</p>
<ol>
- <li><p>For ubuntu you have to add a custom.conf file to /etc/ld.so.conf.d For example,</p>
- <p><code>
- sudo gedit /etc/ld.so.conf.d/randomLibs.conf
- </code></p>
+ <li>
+ <p>For ubuntu you have to add a custom.conf file to /etc/ld.so.conf.d<br>
+ For example,</p>
+ <pre><code>sudo gedit /etc/ld.so.conf.d/randomLibs.conf
+</code></pre>
<p>Inside this file you are supposed to configure the complete path to the directory that
contains all the libraries that you wish to add to the system, let us say
- /home/ubuntu/localLibs</p></li>
- <li><p>To ensure your library location ,check for existence of libthrift.so</p></li>
- <li><p>Save and run the following command to update the system with this libs.</p>
- <p><code>
- sudo ldconfig
- </code></p>
- <p>Note : Remember to add only the path to the directory, not the full path for that file,
- all the libraries inside that path will be automatically indexed.</p></li>
-</ol><h2>Failed to launch the Spark Shell</h2><p><strong>Symptom</strong></p><p>The shell prompts
- the following error :</p><p><code>
- org.apache.spark.sql.CarbonContext$$anon$$apache$spark$sql$catalyst$analysis
- $OverrideCatalog$_setter_$org$apache$spark$sql$catalyst$analysis
- $OverrideCatalog$$overrides_$e
-</code></p><p><strong>Possible Cause</strong></p><p>The Spark Version and the selected Spark Profile
- do not match.</p><p><strong>Procedure</strong></p>
+ /home/ubuntu/localLibs</p>
+ </li>
+ <li>
+ <p>To ensure your library location ,check for existence of <a href="http://libthrift.so">libthrift.so</a>
+ </p>
+ </li>
+ <li>
+ <p>Save and run the following command to update the system with this libs.</p>
+ <pre><code>sudo ldconfig
+</code></pre>
+ </li>
+</ol>
+<pre><code>Note : Remember to add only the path to the directory, not the full path for that file, all the libraries inside that path will be automatically indexed.
+</code></pre>
+<h2 id="failed-to-launch-the-spark-shell">Failed to launch the Spark Shell</h2>
+<p><strong>Symptom</strong></p>
+<p>The shell prompts the following error :</p>
+<pre><code>org.apache.spark.sql.CarbonContext$$anon$$apache$spark$sql$catalyst$analysis
+$OverrideCatalog$_setter_$org$apache$spark$sql$catalyst$analysis
+$OverrideCatalog$$overrides_$e
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The Spark Version and the selected Spark Profile do not match.</p>
+<p><strong>Procedure</strong></p>
<ol>
- <li><p>Ensure your spark version and selected profile for spark are correct.</p></li>
- <li><p>Use the following command :</p>
- <p><code>
- "mvn -Pspark-2.1 -Dspark.version {yourSparkVersion} clean package"
- </code></p>
- <p>Note : Refrain from using "mvn clean package" without specifying the profile.</p></li>
-</ol><h2>Failed to execute load query on cluster.</h2><p><strong>Symptom</strong></p><p>Load query
- failed with the following exception:</p><p><code>
- Dictionary file is locked for updation.
-</code></p><p><strong>Possible Cause</strong></p><p>The carbon.properties file is not identical in
- all the nodes of the cluster.</p><p><strong>Procedure</strong></p><p>Follow the steps to ensure
- the carbon.properties file is consistent across all the nodes:</p>
+ <li>
+ <p>Ensure your spark version and selected profile for spark are correct.</p>
+ </li>
+ <li>
+ <p>Use the following command :</p>
+ </li>
+</ol>
+<pre><code>```
+ "mvn -Pspark-2.1 -Dspark.version {yourSparkVersion} clean package"
+```
+
+Note : Refrain from using "mvn clean package" without specifying the profile.
+</code></pre>
+<h2 id="query-failure-with-generic-error-on-the-beeline">Query Failure with Generic Error
+ on the Beeline</h2>
+<p><strong>Symptom</strong></p>
+<p>Query fails on the executor side and generic error message is printed on the beeline console</p>
+<p><img src="../../../content/docs/latest/images/query_failure_beeline.png?raw=true"
+ alt="Query Failure Beeline"></p>
+<p><strong>Possible Causes</strong></p>
+<ul>
+ <li>In Query flow, Table B-Tree will be loaded into memory on the driver side and filter
+ condition is validated against the min-max of each block to identify false positive,<br>
+ Once the blocks are selected, based on number of available executors, blocks will be
+ distributed to each executor node as shown in below driver logs snapshot
+ </li>
+</ul>
+<p><img src="../../../content/docs/latest/images/query_failure_logs.png?raw=true"
+ alt="Query Failure Logs"></p>
+<ul>
+ <li>
+ <p>When the error occurs in driver side while b-tree loading or block distribution, detail
+ error message will be printed on the beeline console and error trace will be printed on
+ the driver logs.</p>
+ </li>
+ <li>
+ <p>When the error occurs in the executor side, generic error message will be printed as
+ shown in issue description.</p>
+ </li>
+</ul>
+<p><img src="../../../content/docs/latest/images/query_failure_job_details.png?raw=true"
+ alt="Query Failure Job Details"></p>
+<ul>
+ <li>Details of the failed stages can be seen in the Spark Application UI by clicking on the
+ failed stages on the failed job as shown in previous snapshot
+ </li>
+</ul>
+<p><img src="../../../content/docs/latest/images/query_failure_spark_ui.png?raw=true"
+ alt="Query Failure Spark UI"></p>
+<p><strong>Procedure</strong></p>
+<p>Details of the error can be analyzed in details using executor logs available in stdout</p>
+<p><img src="../../../content/docs/latest/images/query_failure_procedure.png?raw=true"
+ alt="Query Failure Spark UI"></p>
+<p>Below snapshot shows executor logs with error message for query failure which can be helpful to
+ locate the error</p>
+<p><img src="../../../content/docs/latest/images/query_failure_issue.png?raw=true"
+ alt="Query Failure Spark UI"></p>
+<h2 id="failed-to-execute-load-query-on-cluster">Failed to execute load query on cluster.
+</h2>
+<p><strong>Symptom</strong></p>
+<p>Load query failed with the following exception:</p>
+<pre><code>Dictionary file is locked for updation.
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The carbon.properties file is not identical in all the nodes of the cluster.</p>
+<p><strong>Procedure</strong></p>
+<p>Follow the steps to ensure the carbon.properties file is consistent across all the nodes:</p>
<ol>
- <li><p>Copy the carbon.properties file from the master node to all the other nodes in the
- cluster. For example, you can use ssh to copy this file to all the nodes.</p></li>
- <li><p>For the changes to take effect, restart the Spark cluster.</p></li>
-</ol><h2>Failed to execute insert query on cluster.</h2><p><strong>Symptom</strong></p><p>Load query
- failed with the following exception:</p><p><code>
- Dictionary file is locked for updation.
-</code></p><p><strong>Possible Cause</strong></p><p>The carbon.properties file is not identical in
- all the nodes of the cluster.</p><p><strong>Procedure</strong></p><p>Follow the steps to ensure
- the carbon.properties file is consistent across all the nodes:</p>
+ <li>
+ <p>Copy the carbon.properties file from the master node to all the other nodes in the
+ cluster.<br>
+ For example, you can use ssh to copy this file to all the nodes.</p>
+ </li>
+ <li>
+ <p>For the changes to take effect, restart the Spark cluster.</p>
+ </li>
+</ol>
+<h2 id="failed-to-execute-insert-query-on-cluster">Failed to execute insert query on
+ cluster.</h2>
+<p><strong>Symptom</strong></p>
+<p>Load query failed with the following exception:</p>
+<pre><code>Dictionary file is locked for updation.
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The carbon.properties file is not identical in all the nodes of the cluster.</p>
+<p><strong>Procedure</strong></p>
+<p>Follow the steps to ensure the carbon.properties file is consistent across all the nodes:</p>
<ol>
- <li><p>Copy the carbon.properties file from the master node to all the other nodes in the
- cluster. For example, you can use scp to copy this file to all the nodes.</p></li>
- <li><p>For the changes to take effect, restart the Spark cluster.</p></li>
-</ol><h2>Failed to connect to hiveuser with thrift</h2><p><strong>Symptom</strong></p><p>We get the
- following exception :</p><p><code>
- Cannot connect to hiveuser.
-</code></p><p><strong>Possible Cause</strong></p><p>The external process does not have permission to
- access.</p><p><strong>Procedure</strong></p><p>Ensure that the Hiveuser in mysql must allow its
- access to the external processes.</p><h2>Failure to read the metastore db during table
- creation.</h2><p><strong>Symptom</strong></p><p>We get the following exception on trying to
- connect :</p><p><code>
- Cannot read the metastore db
-</code></p><p><strong>Possible Cause</strong></p><p>The metastore db is dysfunctional.</p><p>
- <strong>Procedure</strong></p><p>Remove the metastore db from the carbon.metastore in the Spark
- Directory.</p><h2>Failed to load data on the cluster</h2><p><strong>Symptom</strong></p><p>Data
- loading fails with the following exception :</p><p><code>
- Data Load failure exeception
-</code></p><p><strong>Possible Cause</strong></p><p>The following issue can cause the failure :</p>
+ <li>
+ <p>Copy the carbon.properties file from the master node to all the other nodes in the
+ cluster.<br>
+ For example, you can use scp to copy this file to all the nodes.</p>
+ </li>
+ <li>
+ <p>For the changes to take effect, restart the Spark cluster.</p>
+ </li>
+</ol>
+<h2 id="failed-to-connect-to-hiveuser-with-thrift">Failed to connect to hiveuser with
+ thrift</h2>
+<p><strong>Symptom</strong></p>
+<p>We get the following exception :</p>
+<pre><code>Cannot connect to hiveuser.
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The external process does not have permission to access.</p>
+<p><strong>Procedure</strong></p>
+<p>Ensure that the Hiveuser in mysql must allow its access to the external processes.</p>
+<h2 id="failure-to-read-the-metastore-db-during-table-creation">Failure to read the
+ metastore db during table creation.</h2>
+<p><strong>Symptom</strong></p>
+<p>We get the following exception on trying to connect :</p>
+<pre><code>Cannot read the metastore db
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The metastore db is dysfunctional.</p>
+<p><strong>Procedure</strong></p>
+<p>Remove the metastore db from the carbon.metastore in the Spark Directory.</p>
+<h2 id="failed-to-load-data-on-the-cluster">Failed to load data on the cluster</h2>
+<p><strong>Symptom</strong></p>
+<p>Data loading fails with the following exception :</p>
+<pre><code>Data Load failure exeception
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The following issue can cause the failure :</p>
<ol>
- <li><p>The core-site.xml, hive-site.xml, yarn-site and carbon.properties are not consistent
- across all nodes of the cluster.</p></li>
- <li><p>Path to hdfs ddl is not configured correctly in the carbon.properties.</p></li>
-</ol><p><strong>Procedure</strong></p><p>Follow the steps to ensure the following configuration
- files are consistent across all the nodes:</p>
+ <li>
+ <p>The core-site.xml, hive-site.xml, yarn-site and carbon.properties are not consistent
+ across all nodes of the cluster.</p>
+ </li>
+ <li>
+ <p>Path to hdfs ddl is not configured correctly in the carbon.properties.</p>
+ </li>
+</ol>
+<p><strong>Procedure</strong></p>
+<p>Follow the steps to ensure the following configuration files are consistent across all the
+ nodes:</p>
<ol>
- <li><p>Copy the core-site.xml, hive-site.xml, yarn-site,carbon.properties files from the master
- node to all the other nodes in the cluster. For example, you can use scp to copy this file
- to all the nodes.</p>
- <p>Note : Set the path to hdfs ddl in carbon.properties in the master node.</p></li>
- <li><p>For the changes to take effect, restart the Spark cluster.</p></li>
-</ol><h2>Failed to insert data on the cluster</h2><p><strong>Symptom</strong></p><p>Insertion fails
- with the following exception :</p><p><code>
- Data Load failure exeception
-</code></p><p><strong>Possible Cause</strong></p><p>The following issue can cause the failure :</p>
+ <li>
+ <p>Copy the core-site.xml, hive-site.xml, yarn-site,carbon.properties files from the master
+ node to all the other nodes in the cluster.<br>
+ For example, you can use scp to copy this file to all the nodes.</p>
+ <p>Note : Set the path to hdfs ddl in carbon.properties in the master node.</p>
+ </li>
+ <li>
+ <p>For the changes to take effect, restart the Spark cluster.</p>
+ </li>
+</ol>
+<h2 id="failed-to-insert-data-on-the-cluster">Failed to insert data on the cluster</h2>
+<p><strong>Symptom</strong></p>
+<p>Insertion fails with the following exception :</p>
+<pre><code>Data Load failure exeception
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The following issue can cause the failure :</p>
<ol>
- <li><p>The core-site.xml, hive-site.xml, yarn-site and carbon.properties are not consistent
- across all nodes of the cluster.</p></li>
- <li><p>Path to hdfs ddl is not configured correctly in the carbon.properties.</p></li>
-</ol><p><strong>Procedure</strong></p><p>Follow the steps to ensure the following configuration
- files are consistent across all the nodes:</p>
+ <li>
+ <p>The core-site.xml, hive-site.xml, yarn-site and carbon.properties are not consistent
+ across all nodes of the cluster.</p>
+ </li>
+ <li>
+ <p>Path to hdfs ddl is not configured correctly in the carbon.properties.</p>
+ </li>
+</ol>
+<p><strong>Procedure</strong></p>
+<p>Follow the steps to ensure the following configuration files are consistent across all the
+ nodes:</p>
<ol>
- <li><p>Copy the core-site.xml, hive-site.xml, yarn-site,carbon.properties files from the master
- node to all the other nodes in the cluster. For example, you can use scp to copy this file
- to all the nodes.</p>
- <p>Note : Set the path to hdfs ddl in carbon.properties in the master node.</p></li>
- <li><p>For the changes to take effect, restart the Spark cluster.</p></li>
-</ol><h2>Failed to execute Concurrent Operations(Load,Insert,Update) on table by multiple
- workers.</h2><p><strong>Symptom</strong></p><p>Execution fails with the following exception
- :</p><p><code>
- Table is locked for updation.
-</code></p><p><strong>Possible Cause</strong></p><p>Concurrency not supported.</p><p><strong>Procedure</strong>
-</p><p>Worker must wait for the query execution to complete and the table to release the lock for
- another query execution to succeed..</p><h2>Failed to create a table with a single numeric
- column.</h2><p><strong>Symptom</strong></p><p>Execution fails with the following exception :</p>
-<p><code>
- Table creation fails.
-</code></p><p><strong>Possible Cause</strong></p><p>Behavior not supported.</p><p>
- <strong>Procedure</strong></p><p>A single column that can be considered as dimension is
- mandatory for table creation.</p>
\ No newline at end of file
+ <li>
+ <p>Copy the core-site.xml, hive-site.xml, yarn-site,carbon.properties files from the master
+ node to all the other nodes in the cluster.<br>
+ For example, you can use scp to copy this file to all the nodes.</p>
+ <p>Note : Set the path to hdfs ddl in carbon.properties in the master node.</p>
+ </li>
+ <li>
+ <p>For the changes to take effect, restart the Spark cluster.</p>
+ </li>
+</ol>
+<h2 id="failed-to-execute-concurrent-operations">Failed to execute Concurrent Operations.
+</h2>
+<p><strong>Symptom</strong></p>
+<p>Execution of Concurrent Operations (Load,Insert,Update) on table by multiple workers fails with
+ the following exception :</p>
+<pre><code>Table is locked for updation.
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>Concurrency not supported.</p>
+<p><strong>Procedure</strong></p>
+<p>Worker must wait for the query execution to complete and the table to release the lock for
+ another query execution to succeed\u2026</p>
+<h2 id="failed-to-create-a-table-with-a-single-numeric-column">Failed to create a table
+ with a single numeric column.</h2>
+<p><strong>Symptom</strong></p>
+<p>Execution fails with the following exception :</p>
+<pre><code>Table creation fails.
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>Behavior not supported.</p>
+<p><strong>Procedure</strong></p>
+<p>A single column that can be considered as dimension is mandatory for table creation.</p>
+<h2 id="data-failure-because-of-bad-records">Data Failure because of Bad Records</h2>
+<p><strong>Symptom</strong></p>
+<p>Data Loading fails with the following exception</p>
+<pre><code>Error: java.lang.Exception: Data load failed due to Bad record
+</code></pre>
+<p><strong>Possible Causes</strong></p>
+<p>The parameter BAD_RECORDS_ACTION has not been specified in the Query.</p>
+<p><strong>Procedure</strong></p>
+<p>Set the following parameter in the load command OPTIONS as shown below :</p>
+<p>\u2018BAD_RECORDS_ACTION\u2019='FORCE\u2018</p>
+<p><em>Example :</em></p>
+<pre><code>LOAD DATA INPATH 'hdfs://hacluster/user/loader/moredata01.csv' INTO TABLE flow_carbon_256b OPTIONS('DELIMITER'=',', 'BAD_RECORDS_ACTION'='FORCE');
+</code></pre>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/index.html
----------------------------------------------------------------------
diff --git a/content/index.html b/content/index.html
index 9595207..e4048a8 100644
--- a/content/index.html
+++ b/content/index.html
@@ -80,12 +80,13 @@
class="caret"></span></a>
<ul class="dropdown-menu">
<li>
- <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Contributing+to+CarbonData"
+ <a href="https://github.com/apache/incubator-carbondata/blob/master/docs/How-to-contribute-to-Apache-CarbonData.md"
target="_blank">Contributing to CarbonData</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Committers"
target="_blank">Project Committers</a></li>
<li><a href="meetup.html">CarbonData Meetups </a></li>
+ <li><a href="security.html">Apache CarbonData Security</a></li>
</ul>
</li>
<li class="dropdown">
@@ -119,9 +120,29 @@
target="_blank">Thanks</a></li>
</ul>
</li>
+ <li>
+ <a href="#" id="search-icon" ><i class="fa fa-search" aria-hidden="true"></i></a>
+
+ </li>
</ul>
</div><!--/.nav-collapse -->
+ <div id="search-box" >
+ <form method="get" action="http://www.google.com/search">
+ <div class="search-block">
+ <table border="0" cellpadding="0" width="100%">
+ <tr>
+ <td style="width:80%">
+ <input type="text" name="q" size=" 5" maxlength="255" value="" class="search-input" />
+ </td>
+ <td style="width:20%">
+ <input type="submit" value="Search" /></td></tr>
+ <tr><td align="left" style="font-size:75%" colspan="2">
+ <input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked /> Only search for CarbonData </td></tr>
+ </table>
+ </div>
+ </form>
+ </div>
</div>
</nav>
</header> <!-- end Header part -->
@@ -360,9 +381,10 @@
</ul>
<p class="social-icons">
- <a> <i class="fa fa-facebook-square" aria-hidden="true"></i></a>
- <a href="https://twitter.com/search?q=%23CarbonData" target="_blank"> <i
- class="fa fa-twitter-square" aria-hidden="true"></i></a>
+ <a href="https://www.facebook.com/carbondata/" target="_blank">
+ <i class="fa fa-facebook-square" aria-hidden="true"></i></a>
+ <a href="https://twitter.com/search?q=%23CarbonData" target="_blank">
+ <i class="fa fa-twitter-square" aria-hidden="true"></i></a>
<a> <i class="fa fa-linkedin-square" aria-hidden="true"></i></a>
</p>
</div>
@@ -524,7 +546,6 @@
<img src="images/CDincubator.png" alt=""/>
</div>
</div>
-
<div class="col-md-9 col-sm-8">
<div class="cd-apachetxtblock">
<h4 class="title">Apache & OpenSource
@@ -574,7 +595,7 @@
<div class="bottomcontant">
<div class="container-fluid">
<div class="col-md-8 col-sm-8">
- <p class="copyright-txt">Copyright � 2017. All rights reserved |
+ <p class="copyright-txt">Copyright � 2016-2017 Apache CarbonData. All rights reserved |
<a href="http://www.apache.org/" class="term-links">Apache Software
Foundation </a> |
<a href="http://www.apache.org/foundation/policies/privacy.html"
@@ -591,7 +612,6 @@
</div>
</div>
</div>
-
</footer>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/js/custom.js
----------------------------------------------------------------------
diff --git a/content/js/custom.js b/content/js/custom.js
index 81b82ba..72a873c 100644
--- a/content/js/custom.js
+++ b/content/js/custom.js
@@ -63,5 +63,12 @@ $(document).ready(function(){
$("#viewpage").load(pagelink);
});
-
+
+ /*Search box*/
+
+ $(function() {
+ $( "#search-icon" ).click(function() {
+ $( "#search-box" ).toggle();
+ });
+ });
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/meetup.html
----------------------------------------------------------------------
diff --git a/content/meetup.html b/content/meetup.html
index 8e6c74a..b3d1776 100644
--- a/content/meetup.html
+++ b/content/meetup.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
<html lang="en">
- <head>
+<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
@@ -9,163 +9,233 @@
<title>CarbonData:Meetup</title>
<!-- Bootstrap -->
-
+
<link rel="stylesheet" href="css/bootstrap.min.css">
- <link href="css/style.css" rel="stylesheet">
- <link href="css/print.css" rel="stylesheet" >
+ <link href="css/style.css" rel="stylesheet">
+ <link href="css/print.css" rel="stylesheet">
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
<!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
<!--[if lt IE 9]>
- <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
- <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
- </head>
- <body>
- <header>
- <nav class="navbar navbar-default navbar-custom cd-navbar-wrapper" >
- <div class="container">
- <div class="navbar-header">
- <button aria-controls="navbar" aria-expanded="false" data-target="#navbar" data-toggle="collapse" class="navbar-toggle collapsed" type="button">
- <span class="sr-only">Toggle navigation</span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- </button>
- <a href="index.html" class="logo">
- <img src="images/CarbonDataLogo.png" alt="CarbonData logo" title="CarbocnData logo" />
- </a>
+</head>
+<body>
+<header>
+ <nav class="navbar navbar-default navbar-custom cd-navbar-wrapper">
+ <div class="container">
+ <div class="navbar-header">
+ <button aria-controls="navbar" aria-expanded="false" data-target="#navbar"
+ data-toggle="collapse" class="navbar-toggle collapsed" type="button">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a href="index.html" class="logo">
+ <img src="images/CarbonDataLogo.png" alt="CarbonData logo"
+ title="CarbocnData logo"/>
+ </a>
+ </div>
+ <div class="navbar-collapse collapse cd_navcontnt" id="navbar">
+ <ul class="nav navbar-nav navbar-right navlist-custom">
+ <li><a href="index.html" class="hidden-xs"><i class="fa fa-home"
+ aria-hidden="true"></i> </a></li>
+ <li><a href="index.html" class="hidden-lg hidden-md hidden-sm">Home</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button"
+ aria-haspopup="true" aria-expanded="false">Download <span
+ class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/1.0.0-incubating"
+ target="_blank">Apache CarbonData 1.0.0</a></li>
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.2.0-incubating"
+ target="_blank">Apache CarbonData 0.2.0</a></li>
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.1.1-incubating"
+ target="_blank">Apache CarbonData 0.1.1</a></li>
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.1.0-incubating"
+ target="_blank">Apache CarbonData 0.1.0</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
+ target="_blank">Release Archive</a></li>
+ </ul>
+ </li>
+ <li><a href="docs/latest/mainpage.html?page=userguide"
+ class="">Documentation</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button"
+ aria-haspopup="true" aria-expanded="false">Community <span
+ class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://github.com/apache/incubator-carbondata/blob/master/docs/How-to-contribute-to-Apache-CarbonData.md"
+ target="_blank">Contributing to CarbonData</a></li>
+ </li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Committers"
+ target="_blank">Project Committers</a></li>
+ <li><a href="meetup.html">CarbonData Meetups </a></li>
+ <li><a href="security.html">Apache CarbonData Security</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="http://www.apache.org/"
+ class="apache_link hidden-xs dropdown-toggle" data-toggle="dropdown"
+ role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a>
+ </li>
+ <li><a href="http://www.apache.org/licenses/"
+ target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html"
+ target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="http://www.apache.org/"
+ class="hidden-lg hidden-md hidden-sm dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a>
+ </li>
+ <li><a href="http://www.apache.org/licenses/"
+ target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html"
+ target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+ <li>
+ <a href="#" id="search-icon"><i class="fa fa-search" aria-hidden="true"></i></a>
+
+ </li>
+ </ul>
+ </div><!--/.nav-collapse -->
+ <div id="search-box">
+ <form method="get" action="http://www.google.com/search">
+ <div class="search-block">
+ <table border="0" cellpadding="0" width="100%">
+ <tr>
+ <td style="width:80%">
+ <input type="text" name="q" size=" 5" maxlength="255" value=""
+ class="search-input"/>
+ </td>
+ <td style="width:20%">
+ <input type="submit" value="Search"/></td>
+ </tr>
+ <tr>
+ <td align="left" style="font-size:75%" colspan="2">
+ <input type="checkbox" name="sitesearch"
+ value="carbondata.apache.org" checked/> Only search for
+ CarbonData
+ </td>
+ </tr>
+ </table>
+ </div>
+ </form>
+ </div>
</div>
- <div class="navbar-collapse collapse cd_navcontnt" id="navbar">
- <ul class="nav navbar-nav navbar-right navlist-custom">
- <li><a href="index.html" class="hidden-xs"><i class="fa fa-home" aria-hidden="true"></i> </a></li>
- <li><a href="index.html" class="hidden-lg hidden-md hidden-sm">Home</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Download <span class="caret"></span></a>
- <ul class="dropdown-menu">
- <li>
- <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/1.0.0-incubating"
- target="_blank">Apache CarbonData 1.0.0</a></li>
- <li>
- <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.2.0-incubating"
- target="_blank">Apache CarbonData 0.2.0</a></li>
- <li>
- <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.1.1-incubating"
- target="_blank">Apache CarbonData 0.1.1</a></li>
- <li>
- <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.1.0-incubating"
- target="_blank">Apache CarbonData 0.1.0</a></li>
- <li>
- <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
- target="_blank">Release Archive</a></li>
- </ul>
- </li>
- <li><a href="docs/latest/mainpage.html?page=userguide" class="">Documentation</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Community <span class="caret"></span></a>
- <ul class="dropdown-menu">
- <li><a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Contributing+to+CarbonData" target="_blank">Contributing to CarbonData</a></li>
- <li><a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Committers" target="_blank">Project Committers</a></li>
- <li><a href="meetup.html">CarbonData Meetups </a></li>
- </ul>
- </li>
- <li class="dropdown">
- <a href="http://www.apache.org/" class="apache_link hidden-xs dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
- <ul class="dropdown-menu">
- <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
- <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
- <li><a href="http://www.apache.org/foundation/sponsorship.html" target="_blank">Sponsorship</a></li>
- <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
- </ul>
- </li>
- <li class="dropdown">
- <a href="http://www.apache.org/" class="hidden-lg hidden-md hidden-sm dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
- <ul class="dropdown-menu">
- <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
- <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
- <li><a href="http://www.apache.org/foundation/sponsorship.html" target="_blank">Sponsorship</a></li>
- <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
- </ul>
- </li>
- </ul>
- </div><!--/.nav-collapse -->
- </div>
</nav>
- </header> <!-- end Header part -->
-
- <div class="fixed-padding"></div> <!-- top padding with fixde header -->
-
- <section style="position:relative; overflow:hidden ">
+</header> <!-- end Header part -->
+
+<div class="fixed-padding"></div> <!-- top padding with fixde header -->
+
+<section style="position:relative; overflow:hidden ">
<div class="meetup-banber-bg">
- <img src="images/CDbannerSliderbg.jpg" alt="CarbonData banner" />
+ <img src="images/CDbannerSliderbg.jpg" alt="CarbonData banner"/>
</div>
- <div class="meetup-banber">
- <div class="container-fluid">
- <div class="col-md-12 col-sm-12">
- <h1 class="title-heading">ComingUp Meetups
- <span class="title-underline"></span>
- </h1>
- <p class="text-center">
- <span class="txt"> Venue </span>
- <span class="txt"> To be Scheduled </span>
- </p>
-
- <!-- <p class="text-center">
- <a href="#" class="cominguplink"> </a>
- </p>-->
+ <div class="meetup-banber">
+ <div class="container-fluid">
+ <div class="col-md-12 col-sm-12">
+ <h1 class="title-heading">Meetups
+ <span class="title-underline"></span>
+ </h1>
</div>
- </div>
- </div>
- </section>
+ </div>
+ </div>
+</section>
- <section>
- <div class="container-fluid">
- <div class="col-md-12 col-sm-12">
- <div class="meetup-details">
+<section>
+ <div class="container-fluid">
+ <div class="col-md-12 col-sm-12">
+ <div class="meetup-details">
<h2 class="title-heading"> Premiure Events
- <span class="title-underline"></span>
+ <span class="title-underline"></span>
</h2>
<div class="meetup-info">
- <div class="row">
- <div class="col-md-10 col-sm-9">
- <h2 class="meetup-title">Oscon - Austin TX<span class="meetup-date">May 2016</span></h2>
- <!--<p class="meetup-location"> </p>-->
- <p>The agenda for the meetup was 'Carbondata : A New Hadoop File Format for Faster Data Analysis'. It focused on the inception and introduction of CarbonData to the open source community.</p>
- </div>
- <div class="col-md-2 col-sm-3">
- <div class="meetup-icon">
- <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=66850609" target="blank">
- <i class="fa fa-file-pdf-o" aria-hidden="true"></i>
- </a>
-
+ <div class="row">
+ <div class="col-md-10 col-sm-9">
+ <h2 class="meetup-title">Oscon - Austin TX<span class="meetup-date">May 2016</span>
+ </h2>
+ <!--<p class="meetup-location"> </p>-->
+ <p>The agenda for the meetup was 'Carbondata : A New Hadoop File Format
+ for Faster Data Analysis'. It focused on the inception and
+ introduction of CarbonData to the open source community.</p>
+ </div>
+ <div class="col-md-2 col-sm-3">
+ <div class="meetup-icon">
+ <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=66850609"
+ target="blank">
+ <i class="fa fa-angle-right" aria-hidden="true"></i>
+ </a>
+ </div>
+ </div>
</div>
- </div>
- </div>
</div>
<div class="meetup-info">
- <div class="row">
- <div class="col-md-10 col-sm-9">
- <h2 class="meetup-title">Beijing<span class="meetup-date">Oct 2016</span></h2>
- <!-- <p class="meetup-location"> </p>-->
- <p>The agenda for the meetup was 'CarbonData : An Indexed Columnar File Format for Interactive query'. It detailedly introduced the architecture and techniques of Apache CarbonData ,followed by a tutorial session and open discussion about future plans of Apache Carbondata.</p>
- </div>
- <div class="col-md-2 col-sm-3">
- <div class="meetup-icon">
- <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/10.29+Beijing+Apache+CarbonData+Meetup+PPT" target="blank">
- <i class="fa fa-file-pdf-o" aria-hidden="true"></i>
- </a>
-
-
-
+ <div class="row">
+ <div class="col-md-10 col-sm-9">
+ <h2 class="meetup-title">Beijing<span
+ class="meetup-date">Oct 29, 2016</span></h2>
+ <!-- <p class="meetup-location"> </p>-->
+ <p>The agenda for the meetup was 'CarbonData : An Indexed Columnar File
+ Format for Interactive query'. It detailedly introduced the
+ architecture and techniques of Apache CarbonData ,followed by a
+ tutorial session and open discussion about future plans of Apache
+ Carbondata.</p>
</div>
- </div>
- </div>
+ <div class="col-md-2 col-sm-3">
+ <div class="meetup-icon">
+ <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=66849573"
+ target="blank">
+ <i class="fa fa-angle-right" aria-hidden="true"></i>
+ </a>
+ </div>
+ </div>
+ </div>
+ </div>
+ <div class="meetup-info">
+ <div class="row">
+ <div class="col-md-10 col-sm-9">
+ <h2 class="meetup-title">Spark Summit East 2017-Apache CarbonData<span class="meetup-date">Feb 8, 2017</span>
+ </h2>
+ <!--<p class="meetup-location"> </p>-->
+ <p>The agenda for the meetup was 'Apache Carbondata: An Indexed Columnar File Format for Interactive Query with Spark SQL'</p>
+ </div>
+ <div class="col-md-2 col-sm-3">
+ <div class="meetup-icon">
+ <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=68714802"
+ target="blank">
+ <i class="fa fa-angle-right" aria-hidden="true"></i>
+ </a>
+ </div>
+ </div>
+ </div>
</div>
- </div>
</div>
- </div>
- </section>
-
+ </div>
+ </div>
+</section>
+
<!--
<footer>
<div class="topcontant">
@@ -205,11 +275,11 @@
</div>
</footer> -->
- <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+<!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
- <script src="js/jquery.min.js"></script>
- <!-- Include all compiled plugins (below), or include individual files as needed -->
- <script src="js/bootstrap.min.js"></script>
- <script src="js/custom.js"></script>
- </body>
- </html>
\ No newline at end of file
+<script src="js/jquery.min.js"></script>
+<!-- Include all compiled plugins (below), or include individual files as needed -->
+<script src="js/bootstrap.min.js"></script>
+<script src="js/custom.js"></script>
+</body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/pdf/maven-pdf-plugin.pdf
----------------------------------------------------------------------
diff --git a/content/pdf/maven-pdf-plugin.pdf b/content/pdf/maven-pdf-plugin.pdf
index 64ae580..881f80b 100644
Binary files a/content/pdf/maven-pdf-plugin.pdf and b/content/pdf/maven-pdf-plugin.pdf differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/content/security.html
----------------------------------------------------------------------
diff --git a/content/security.html b/content/security.html
new file mode 100644
index 0000000..205eba4
--- /dev/null
+++ b/content/security.html
@@ -0,0 +1,173 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href='images/favicon.ico' rel='shortcut icon' type='image/x-icon'>
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>CarbonData:Security</title>
+
+ <!-- Bootstrap -->
+
+ <link rel="stylesheet" href="css/bootstrap.min.css">
+ <link href="css/style.css" rel="stylesheet">
+ <link href="css/print.css" rel="stylesheet" >
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+</head>
+<body>
+<header>
+ <nav class="navbar navbar-default navbar-custom cd-navbar-wrapper" >
+ <div class="container">
+ <div class="navbar-header">
+ <button aria-controls="navbar" aria-expanded="false" data-target="#navbar" data-toggle="collapse" class="navbar-toggle collapsed" type="button">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a href="index.html" class="logo">
+ <img src="images/CarbonDataLogo.png" alt="CarbonData logo" title="CarbocnData logo" />
+ </a>
+ </div>
+ <div class="navbar-collapse collapse cd_navcontnt" id="navbar">
+ <ul class="nav navbar-nav navbar-right navlist-custom">
+ <li><a href="index.html" class="hidden-xs"><i class="fa fa-home" aria-hidden="true"></i> </a></li>
+ <li><a href="index.html" class="hidden-lg hidden-md hidden-sm">Home</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Download <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/1.0.0-incubating"
+ target="_blank">Apache CarbonData 1.0.0</a></li>
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.2.0-incubating"
+ target="_blank">Apache CarbonData 0.2.0</a></li>
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.1.1-incubating"
+ target="_blank">Apache CarbonData 0.1.1</a></li>
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.1.0-incubating"
+ target="_blank">Apache CarbonData 0.1.0</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
+ target="_blank">Release Archive</a></li>
+ </ul>
+ </li>
+ <li><a href="docs/latest/mainpage.html?page=userguide" class="">Documentation</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Community <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://github.com/apache/incubator-carbondata/blob/master/docs/How-to-contribute-to-Apache-CarbonData.md"
+ target="_blank">Contributing to CarbonData</a></li></li>
+ <li><a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Committers" target="_blank">Project Committers</a></li>
+ <li><a href="meetup.html">CarbonData Meetups </a></li>
+ <li><a href="security.html">Apache CarbonData Security </a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="apache_link hidden-xs dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html" target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="hidden-lg hidden-md hidden-sm dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html" target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+ <li>
+ <a href="#" id="search-icon" ><i class="fa fa-search" aria-hidden="true"></i></a>
+ </li>
+ </ul>
+ </div><!--/.nav-collapse -->
+
+ <div id="search-box" >
+ <form method="get" action="http://www.google.com/search">
+ <div class="search-block">
+ <table border="0" cellpadding="0" width="100%">
+ <tr>
+ <td style="width:80%">
+ <input type="text" name="q" size=" 5" maxlength="255" value="" class="search-input" />
+ </td>
+ <td style="width:20%">
+ <input type="submit" value="Search" /></td></tr>
+ <tr><td align="left" style="font-size:75%" colspan="2">
+ <input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked /> Only search for CarbonData </td></tr>
+ </table>
+ </div>
+ </form>
+ </div>
+
+ </div>
+ </nav>
+</header> <!-- end Header part -->
+
+<div class="fixed-padding"></div> <!-- top padding with fixde header -->
+
+<section style="position:relative; overflow:hidden ">
+ <div class="meetup-banber-bg">
+ <img src="images/CDbannerSliderbg.jpg" alt="CarbonData banner" />
+ </div>
+
+ <div class="meetup-banber" style="padding:30px">
+ <div class="container-fluid">
+ <div class="col-md-12 col-sm-12">
+ <h1 class="title-heading">
+ Indexed Columnar Data Format for Faster Analytics on <br>Big Data Platforms.
+
+ <span class="title-underline"></span>
+ </h1>
+ </div>
+ </div>
+ </div>
+</section>
+
+<section>
+ <div class="container">
+ <div class="row">
+ <div class="col-md-12 col-sm-12">
+ <div class="security-block">
+ <h1 class="title-heading"> Apache CarbonData Security
+ <span class="title-underline"></span>
+ </h1>
+
+ <p>
+ The Apache Software Foundation takes a rigorous standpoint in annihilating the security issues in its software projects. Apache CarbonData is highly sensitive and forthcoming to issues pertaining to its features and functionality.
+ </p>
+ <p>If you have apprehensions regarding CarbonData\u2019s security or you discover vulnerability or potential threat, don\u2019t hesitate to get in touch with the Apache Security Team by dropping a mail at <a href="mailto:security@apache.org"> security@apache.org</a>. In the mail, specify the project name CarbonData with the description of the issue or potential threat. You are also urged to recommend the way to reproduce and replicate the issue. The security team and the CarbonData community will get back to you after assessing and analysing the findings.</p>
+ <p>
+ PLEASE PAY ATTENTION to report the security issue on the security email before disclosing it on public domain.
+ </p>
+ <p>
+ The ASF Security Team maintains a page with the description of how vulnerabilities and potential threats are handled, check their <a href="http://www.apache.org/security/" target="_blank">Web page</a> for more Details.
+ </p>
+
+ </div>
+ </div>
+ </div>
+ </div>
+</section>
+
+
+<!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+
+<script src="js/jquery.min.js"></script>
+<!-- Include all compiled plugins (below), or include individual files as needed -->
+<script src="js/bootstrap.min.js"></script>
+<script src="js/custom.js"></script>
+<script src="js/mdNavigation.js" type="text/javascript"></script>
+</body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/main/webapp/css/style.css
----------------------------------------------------------------------
diff --git a/src/main/webapp/css/style.css b/src/main/webapp/css/style.css
index 3540959..2aa6248 100644
--- a/src/main/webapp/css/style.css
+++ b/src/main/webapp/css/style.css
@@ -840,15 +840,15 @@ p img{ max-width: 100% }
.meetup-info .meetup-location { font-size:16px; color: #444; margin-bottom: 10px;}
.meetup-info p { font-size:13px; color: #444; line-height: 18px; margin: 0 }
-.meetup-info i {font-size: 60px; color: #ff943d; border: 1px dashed #ff943d; padding: 15px; margin-bottom:6px; background-color: #fff;}
-.meetup-info:hover{background-color: #fff; border-color: #ff943d;}
+.meetup-info i {font-size: 60px; color: #ff943d; /*border: 1px dashed #ff943d;*/ padding: 15px; margin-bottom:6px; background-color: #fff;}
+.meetup-info:hover{background-color: #fff; border-color: #ff943d;}
.meetup-info .viewlink{ font-size:14px;}
.meetup-info .meetup-icon{ text-align: right}
.meetup-info .viewlink {font-size: 14px; width: 84px; background: #ff943d; display: inline-block; text-align: center;
padding: 4px 2px; color: #fff; font-weight: 600;}
.meetup-info .viewlink:hover {background: #dd6300;}
-.meetup-info i:hover {color: #dd6300; border: 1px dashed #dd6300; }
+.meetup-info i:hover {color: #dd6300; /*border: 1px dashed #dd6300; */}
.feedbackbox{width:80%; margin:20px auto; padding:20px; border:1px solid #eee; background-color: #fff;
box-shadow: 0 3px 2px #aaa }
@@ -902,6 +902,41 @@ box-shadow: 0 3px 2px #aaa }
height: 36px;
}
+#search-box{ position: relative; display: none;}
+.search-block{
+ border: 5px solid #ff943d;
+ padding: 6px;
+ width: 24em;
+ position: absolute;
+ background: #f8f8f8;
+ right: 0;
+
+}
+
+.search-block input[type="text"] {
+ width: 244px;
+ border: 1px solid #ff9443;
+ padding: 0 5px;
+}
+.search-block input[type="submit"] {
+ width: 70px;
+ color: white;
+ background: #ff943d;
+ border: 1px solid #ff943d;
+ font-size: 14px;
+
+}
+
+.search-block input:focus{ border: 1px solid #dd6300;}
+
+ .security-block {background: #fff; margin: 15px 0; padding:5px 15px;}
+ .security-block p{ margin:0 5px 20px;
+ font-size: 14px;
+ color: #444;
+ line-height: 20px;
+ display: block;
+ text-align: justify}
+
/* ------------------- End Documentation Style here*/
@@ -967,7 +1002,11 @@ box-shadow: 0 3px 2px #aaa }
}
+/* Media Close*/
+
+.mobile{ display: none;}
+.desktop{ display: block;}
/* Media Close*/
[2/4] incubator-carbondata-site git commit: updated the FAQs and
added Security page and Search Functionality
Posted by ch...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/main/webapp/docs/latest/faq.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/docs/latest/faq.html b/src/main/webapp/docs/latest/faq.html
index 3a7a354..d817823 100644
--- a/src/main/webapp/docs/latest/faq.html
+++ b/src/main/webapp/docs/latest/faq.html
@@ -15,13 +15,188 @@
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
---><h1>FAQs</h1>
+-->
+<h1 id="faqs">FAQs</h1>
<ul>
- <li><p><strong>Auto Compaction not Working</strong></p><p>The Property carbon.enable.auto.load.merge in carbon.properties need to be set to true.</p></li>
- <li><p><strong>Getting Abstract method error</strong></p><p>You need to specify the spark version while using Maven to build project.</p></li>
- <li><p><strong>Getting NotImplementedException for subquery using IN and EXISTS</strong></p><p>Subquery with in and exists not supported in CarbonData.</p></li>
- <li><p><strong>Getting Exceptions on creating a view</strong></p><p>View not supported in CarbonData.</p></li>
- <li><p><strong>How to verify if ColumnGroups have been created as desired.</strong></p><p>Try using desc table query.</p></li>
- <li><p><strong>Did anyone try to run CarbonData on windows? Is it supported on Windows?</strong></p><p>We may provide support for windows in future. You are welcome to contribute if you want to add the support :)</p></li>
- <li><p><strong>Can we execute Concurrent Operations(Load,Insert,Update) on table by multiple workers.</strong></p><p>Concurrency is not supported in current release of CarbonData.</p></li>
-</ul>
\ No newline at end of file
+ <li>
+ <p><a href="#can-we-preserve-segments-from-compaction">Can we preserve Segments from
+ Compaction?</a></p>
+ </li>
+ <li>
+ <p><a href="#can-we-disable-horizontal-compaction">Can we disable horizontal compaction?</a></p>
+ </li>
+ <li>
+ <p><a href="#what-is-horizontal-compaction">What is horizontal compaction?</a></p>
+ </li>
+ <li>
+ <p><a href="#how-to-enable-compaction-while-data-loading">How to enable Compaction while data
+ loading?</a></p>
+ </li>
+ <li>
+ <p><a href="#where-are-bad-records-stored-in-carbondata">Where are Bad Records Stored in
+ CarbonData?</a></p>
+ </li>
+ <li>
+ <p><a href="#what-are-bad-records">What are Bad Records?</a></p>
+ </li>
+ <li>
+ <p><a href="#can-we-use-carbondata-on-standalone-spark-cluster">Can we use CarbonData on
+ Standalone Spark Cluster?</a></p>
+ </li>
+ <li>
+ <p><a href="#what-versions-of-apache-spark-are-compatible-with-carbondata">What versions of
+ Apache Spark are Compatible with CarbonData?</a></p>
+ </li>
+ <li>
+ <p><a href="#can-we-load-data-from-excel">Can we Load Data from excel?</a></p>
+ </li>
+ <li>
+ <p><a href="#how-to-enable-single-pass-data-loading">How to enable Single Pass Data Loading?</a>
+ </p>
+ </li>
+ <li>
+ <p><a href="#what-is-single-pass-data-loading">What is Single Pass Data Loading?</a></p>
+ </li>
+ <li>
+ <p><a href="#how-to-specify-the-data-loading-format-for-carbondata">How to specify the data
+ loading format for CarbonData ?</a></p>
+ </li>
+ <li>
+ <p><a href="#how-to-resolve-store-location-can-not-be-found">How to resolve store location can\u2019t
+ be found?</a></p>
+ </li>
+ <li>
+ <p><a href="">What is carbon.lock.type?</a></p>
+ </li>
+ <li>
+ <p><a href="#how-to-enable-auto-compaction">How to enable Auto Compaction?</a></p>
+ </li>
+ <li>
+ <p><a href="#how-to-resolve-abstract-method-error">How to resolve Abstract Method Error?</a></p>
+ </li>
+ <li>
+ <p><a href="#getting-exception-on-creating-a-view">Getting Exception on Creating a View</a></p>
+ </li>
+ <li>
+ <p><a href="#is-carbondata-supported-for-windows">Is CarbonData supported for Windows?</a></p>
+ </li>
+
+</ul>
+
+<h2 id="can-we-preserve-segments-from-compaction">Can we preserve Segments from Compaction?</h2>
+<p>If you want to preserve number of segments from being compacted then you can set the property
+ <strong>carbon.numberof.preserve.segments</strong> equal to the <strong>value of number of
+ segments to be preserved</strong>.</p>
+<p>Note : <em>No segments are preserved by Default.</em></p>
+
+<h2 id="can-we-disable-horizontal-compaction">Can we disable horizontal compaction?</h2>
+<p>Yes, to disable horizontal compaction, set <strong>carbon.horizontal.compaction.enable</strong>
+ to <code>FALSE</code> in carbon.properties file.</p>
+
+<h2 id="what-is-horizontal-compaction">What is horizontal compaction?</h2>
+<p>Compaction performed after Update and Delete operations is referred as Horizontal Compaction.
+ After every DELETE and UPDATE operation, horizontal compaction may occur in case the delta
+ (DELETE/ UPDATE) files becomes more than specified threshold.</p>
+<p>By default the parameter <strong>carbon.horizontal.compaction.enable</strong> enabling the
+ horizontal compaction is set to <code>TRUE</code>.</p>
+
+<h2 id="how-to-enable-compaction-while-data-loading">How to enable Compaction while data
+ loading?</h2>
+<p>To enable compaction while data loading, set <strong>carbon.enable.auto.load.merge</strong> to
+ <code>TRUE</code> in carbon.properties file.</p>
+
+<h2 id="where-are-bad-records-stored-in-carbondata">Where are Bad Records Stored in CarbonData?</h2>
+<p>The bad records are stored at the location set in carbon.badRecords.location in carbon.properties
+ file.<br>
+ By default <strong>carbon.badRecords.location</strong> specifies the following location <code>/opt/Carbon/Spark/badrecords</code>.
+</p>
+
+<h2 id="what-are-bad-records">What are Bad Records?</h2>
+<p>Records that fail to get loaded into the CarbonData due to data type incompatibility are
+ classified as Bad Records.</p>
+
+<h2 id="can-we-use-carbondata-on-standalone-spark-cluster">Can we use CarbonData on Standalone Spark
+ Cluster?</h2>
+<p>Yes, CarbonData can be used on a Standalone spark cluster. But using a standalone cluster has
+ following limitations:</p>
+<ul>
+ <li>single node cluster cannot be scaled up</li>
+ <li>the maximum memory and the CPU computation power has a fixed limit</li>
+ <li>the number of processors are limited in a single node cluster</li>
+</ul>
+<p>To harness the actual speed of execution of CarbonData on petabytes of data, it is suggested to
+ use a Multinode Cluster.</p>
+
+<h2 id="what-versions-of-apache-spark-are-compatible-with-carbondata">What versions of Apache Spark
+ are Compatible with CarbonData?</h2>
+<p>Currently <strong>Spark 1.6.2</strong> and <strong>Spark 2.1</strong> is compatible with
+ CarbonData.</p>
+
+<h2 id="can-we-load-data-from-excel">Can we Load Data from excel?</h2>
+<p>Yes, the data can be loaded from excel provided the data is in CSV format.</p>
+
+<h2 id="how-to-enable-single-pass-data-loading">How to enable Single Pass Data Loading?</h2>
+<p>You need to set <strong>SINGLE_PASS</strong> to <code>True</code> and append it to
+ <code>OPTIONS</code> Section in the query as demonstrated in the Load Query below :</p>
+<pre><code>LOAD DATA local inpath '/opt/rawdata/data.csv' INTO table carbontable
+OPTIONS('DELIMITER'=',', 'QUOTECHAR'='"','FILEHEADER'='empno,empname,designation','USE_KETTLE'='FALSE')
+</code></pre>
+<p>Refer to <a
+ href="https://github.com/PallaviSingh1992/incubator-carbondata/blob/6b4dd5f3dea8c93839a94c2d2c80ab7a799cf209/docs/dml-operation-on-carbondata.md">DML-operations-in-CarbonData</a>
+ for more details and example.</p>
+
+<h2 id="what-is-single-pass-data-loading">What is Single Pass Data Loading?</h2>
+<p>Single Pass Loading enables single job to finish data loading with dictionary generation on the
+ fly. It enhances performance in the scenarios where the subsequent data loading after initial
+ load involves fewer incremental updates on the dictionary.<br>
+ This option specifies whether to use single pass for loading data or not. By default this option
+ is set to <code>FALSE</code>.</p>
+
+<h2 id="how-to-specify-the-data-loading-format-for-carbondata">How to specify the data loading
+ format for CarbonData?</h2>
+<p>Edit carbon.properties file. Modify the value of parameter
+ <strong>carbon.data.file.version</strong>.<br>
+ Setting the parameter <strong>carbon.data.file.version</strong> to <code>1</code> will support
+ data loading in <code>old format(0.x version)</code> and setting <strong>carbon.data.file.version</strong>
+ to <code>2</code> will support data loading in <code>new format(1.x onwards)</code> only.<br>
+ By default the data loading is supported using the new format.</p>
+
+<h2 id="how-to-resolve-store-location-can-not-be-found">How to resolve store location can not be
+ found?</h2>
+<p>Try creating <code>carbonsession</code> with <code>storepath</code> specified in the following
+ manner :</p>
+<pre><code>val carbon = SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession(<store_path>)
+</code></pre>
+<p>Example:</p>
+<pre><code>val carbon = SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession("hdfs://localhost:9000/carbon/store ")
+</code></pre>
+
+<h2 id="what-is-carbon-lockt-ype">What is carbon.lock.type?</h2>
+<p>This property configuration specifies the type of lock to be acquired during concurrent
+ operations on table. This property can be set with the following values :</p>
+<ul>
+ <li><strong>LOCALLOCK</strong> : This Lock is created on local file system as file. This lock is
+ useful when only one spark driver (thrift server) runs on a machine and no other CarbonData
+ spark application is launched concurrently.
+ </li>
+ <li><strong>HDFSLOCK</strong> : This Lock is created on HDFS file system as file. This lock is
+ useful when multiple CarbonData spark applications are launched and no ZooKeeper is running
+ on cluster and the HDFS supports, file based locking.
+ </li>
+</ul>
+
+<h2 id="how-to-enable-auto-compaction">How to enable Auto Compaction?</h2>
+<p>To enable compaction set <strong>carbon.enable.auto.load.merge</strong> to <code>TRUE</code> in
+ the carbon.properties file.</p>
+
+<h2 id="how-to-resolve-abstract-method-error">How to resolve Abstract Method Error?</h2>
+<p>You need to specify the <code>spark version</code> while using Maven to build project.</p>
+
+<h2 id="getting-exception-on-creating-a-view">Getting Exception on Creating a View</h2>
+<p>View not supported in CarbonData.</p>
+
+<h2 id="is-carbondata-supported-for-windows">Is CarbonData supported for Windows?</h2>
+<p>We may provide support for windows in future. You are welcome to contribute if you want to add
+ the support :)</p>
+
+</body></html>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/main/webapp/docs/latest/images/query_failure_beeline.png
----------------------------------------------------------------------
diff --git a/src/main/webapp/docs/latest/images/query_failure_beeline.png b/src/main/webapp/docs/latest/images/query_failure_beeline.png
new file mode 100644
index 0000000..e4ec22b
Binary files /dev/null and b/src/main/webapp/docs/latest/images/query_failure_beeline.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/main/webapp/docs/latest/images/query_failure_issue.png
----------------------------------------------------------------------
diff --git a/src/main/webapp/docs/latest/images/query_failure_issue.png b/src/main/webapp/docs/latest/images/query_failure_issue.png
new file mode 100644
index 0000000..87270d2
Binary files /dev/null and b/src/main/webapp/docs/latest/images/query_failure_issue.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/main/webapp/docs/latest/images/query_failure_job_details.png
----------------------------------------------------------------------
diff --git a/src/main/webapp/docs/latest/images/query_failure_job_details.png b/src/main/webapp/docs/latest/images/query_failure_job_details.png
new file mode 100644
index 0000000..26e607d
Binary files /dev/null and b/src/main/webapp/docs/latest/images/query_failure_job_details.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/main/webapp/docs/latest/images/query_failure_logs.png
----------------------------------------------------------------------
diff --git a/src/main/webapp/docs/latest/images/query_failure_logs.png b/src/main/webapp/docs/latest/images/query_failure_logs.png
new file mode 100644
index 0000000..8fbdfa6
Binary files /dev/null and b/src/main/webapp/docs/latest/images/query_failure_logs.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/main/webapp/docs/latest/images/query_failure_procedure.png
----------------------------------------------------------------------
diff --git a/src/main/webapp/docs/latest/images/query_failure_procedure.png b/src/main/webapp/docs/latest/images/query_failure_procedure.png
new file mode 100644
index 0000000..9d2c81f
Binary files /dev/null and b/src/main/webapp/docs/latest/images/query_failure_procedure.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/main/webapp/docs/latest/images/query_failure_spark_ui.png
----------------------------------------------------------------------
diff --git a/src/main/webapp/docs/latest/images/query_failure_spark_ui.png b/src/main/webapp/docs/latest/images/query_failure_spark_ui.png
new file mode 100644
index 0000000..1802760
Binary files /dev/null and b/src/main/webapp/docs/latest/images/query_failure_spark_ui.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/main/webapp/docs/latest/mainpage.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/docs/latest/mainpage.html b/src/main/webapp/docs/latest/mainpage.html
index b8d0346..7e62350 100644
--- a/src/main/webapp/docs/latest/mainpage.html
+++ b/src/main/webapp/docs/latest/mainpage.html
@@ -71,9 +71,11 @@
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Community <span class="caret"></span></a>
<ul class="dropdown-menu">
- <li><a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Contributing+to+CarbonData" target="_blank">Contributing to CarbonData</a></li>
+ <li><a href="https://github.com/apache/incubator-carbondata/blob/master/docs/How-to-contribute-to-Apache-CarbonData.md"
+ target="_blank">Contributing to CarbonData</a></li></li>
<li><a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Committers" target="_blank">Project Committers</a></li>
<li><a href="../../meetup.html">CarbonData Meetups </a></li>
+ <li><a href="../../security.html">Apache CarbonData Security</a></li>
</ul>
</li>
<li class="dropdown">
@@ -96,8 +98,29 @@
</ul>
</li>
+ <li>
+ <a href="#" id="search-icon" ><i class="fa fa-search" aria-hidden="true"></i></a>
+
+ </li>
+
</ul>
</div><!--/.nav-collapse -->
+ <div id="search-box" >
+ <form method="get" action="http://www.google.com/search">
+ <div class="search-block">
+ <table border="0" cellpadding="0" width="100%">
+ <tr>
+ <td style="width:80%">
+ <input type="text" name="q" size=" 5" maxlength="255" value="" class="search-input" />
+ </td>
+ <td style="width:20%">
+ <input type="submit" value="Search" /></td></tr>
+ <tr><td align="left" style="font-size:75%" colspan="2">
+ <input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked /> Only search for CarbonData </td></tr>
+ </table>
+ </div>
+ </form>
+ </div>
</div>
</nav>
</header> <!-- end Header part -->
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/main/webapp/docs/latest/troubleshooting.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/docs/latest/troubleshooting.html b/src/main/webapp/docs/latest/troubleshooting.html
index a3fb1a5..4416f16 100644
--- a/src/main/webapp/docs/latest/troubleshooting.html
+++ b/src/main/webapp/docs/latest/troubleshooting.html
@@ -15,116 +15,283 @@
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
---><h1>Troubleshooting</h1><p>This tutorial is designed to provide troubleshooting for end users and
- developers who are building, deploying, and using CarbonData.</p><h2>Failed to load thrift
- libraries</h2><p><strong>Symptom</strong></p><p>Thrift throws following exception :</p><p><code>
- thrift: error while loading shared libraries:
- libthriftc.so.0: cannot open shared object file: No such file or directory
-</code></p><p><strong>Possible Cause</strong></p><p>The complete path to the directory containing
- the libraries is not configured correctly.</p><p><strong>Procedure</strong></p><p>Follow the
- steps below to ensure loading of libraries appropriately :</p>
+--><h1>Troubleshooting</h1>
+<p>This tutorial is designed to provide troubleshooting for end users and developers
+ who are building, deploying, and using CarbonData.</p>
+<ul>
+ <li><a href="#failed-to-load-thrift-libraries">Failed to load thrift libraries</a></li>
+ <li><a href="#failed-to-launch-the-spark-shell">Failed to launch the Spark Shell</a></li>
+ <li><a href="#query-failure-with-generic-error-on-the-beeline">Query Failure with Generic Error
+ on the Beeline</a></li>
+ <li><a href="#failed-to-execute-load-query-on-cluster">Failed to execute load query on
+ cluster</a></li>
+ <li><a href="#failed-to-execute-insert-query-on-cluster">Failed to execute insert query on
+ cluster</a></li>
+ <li><a href="#failed-to-connect-to-hiveuser-with-thrift">Failed to connect to hiveuser with
+ thrift</a></li>
+ <li><a href="#failure-to-read-the-metastore-db-during-table-creation">Failure to read the
+ metastore db during table creation</a></li>
+ <li><a href="#failed-to-load-data-on-the-cluster">Failed to load data on the cluster</a></li>
+ <li><a href="#failed-to-insert-data-on-the-cluster">Failed to insert data on the cluster</a>
+ </li>
+ <li><a href="#failed-to-execute-concurrent-operations">Failed to execute Concurrent
+ Operations</a></li>
+ <li><a href="#failed-to-create-a-table-with-a-single-numeric-column">Failed to create a table
+ with a single numeric column</a></li>
+ <li><a href="#data-failure-because-of-bad-records">Data Failure because of Bad Records</a></li>
+</ul>
+<h2 id="failed-to-load-thrift-libraries">Failed to load thrift libraries</h2>
+<p><strong>Symptom</strong></p>
+<p>Thrift throws following exception :</p>
+<pre><code>thrift: error while loading shared libraries:
+libthriftc.so.0: cannot open shared object file: No such file or directory
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The complete path to the directory containing the libraries is not configured correctly.</p>
+<p><strong>Procedure</strong></p>
+<p>Follow the steps below to ensure loading of libraries appropriately :</p>
<ol>
- <li><p>For ubuntu you have to add a custom.conf file to /etc/ld.so.conf.d For example,</p>
- <p><code>
- sudo gedit /etc/ld.so.conf.d/randomLibs.conf
- </code></p>
+ <li>
+ <p>For ubuntu you have to add a custom.conf file to /etc/ld.so.conf.d<br>
+ For example,</p>
+ <pre><code>sudo gedit /etc/ld.so.conf.d/randomLibs.conf
+</code></pre>
<p>Inside this file you are supposed to configure the complete path to the directory that
contains all the libraries that you wish to add to the system, let us say
- /home/ubuntu/localLibs</p></li>
- <li><p>To ensure your library location ,check for existence of libthrift.so</p></li>
- <li><p>Save and run the following command to update the system with this libs.</p>
- <p><code>
- sudo ldconfig
- </code></p>
- <p>Note : Remember to add only the path to the directory, not the full path for that file,
- all the libraries inside that path will be automatically indexed.</p></li>
-</ol><h2>Failed to launch the Spark Shell</h2><p><strong>Symptom</strong></p><p>The shell prompts
- the following error :</p><p><code>
- org.apache.spark.sql.CarbonContext$$anon$$apache$spark$sql$catalyst$analysis
- $OverrideCatalog$_setter_$org$apache$spark$sql$catalyst$analysis
- $OverrideCatalog$$overrides_$e
-</code></p><p><strong>Possible Cause</strong></p><p>The Spark Version and the selected Spark Profile
- do not match.</p><p><strong>Procedure</strong></p>
+ /home/ubuntu/localLibs</p>
+ </li>
+ <li>
+ <p>To ensure your library location ,check for existence of <a href="http://libthrift.so">libthrift.so</a>
+ </p>
+ </li>
+ <li>
+ <p>Save and run the following command to update the system with this libs.</p>
+ <pre><code>sudo ldconfig
+</code></pre>
+ </li>
+</ol>
+<pre><code>Note : Remember to add only the path to the directory, not the full path for that file, all the libraries inside that path will be automatically indexed.
+</code></pre>
+<h2 id="failed-to-launch-the-spark-shell">Failed to launch the Spark Shell</h2>
+<p><strong>Symptom</strong></p>
+<p>The shell prompts the following error :</p>
+<pre><code>org.apache.spark.sql.CarbonContext$$anon$$apache$spark$sql$catalyst$analysis
+$OverrideCatalog$_setter_$org$apache$spark$sql$catalyst$analysis
+$OverrideCatalog$$overrides_$e
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The Spark Version and the selected Spark Profile do not match.</p>
+<p><strong>Procedure</strong></p>
<ol>
- <li><p>Ensure your spark version and selected profile for spark are correct.</p></li>
- <li><p>Use the following command :</p>
- <p><code>
- "mvn -Pspark-2.1 -Dspark.version {yourSparkVersion} clean package"
- </code></p>
- <p>Note : Refrain from using "mvn clean package" without specifying the profile.</p></li>
-</ol><h2>Failed to execute load query on cluster.</h2><p><strong>Symptom</strong></p><p>Load query
- failed with the following exception:</p><p><code>
- Dictionary file is locked for updation.
-</code></p><p><strong>Possible Cause</strong></p><p>The carbon.properties file is not identical in
- all the nodes of the cluster.</p><p><strong>Procedure</strong></p><p>Follow the steps to ensure
- the carbon.properties file is consistent across all the nodes:</p>
+ <li>
+ <p>Ensure your spark version and selected profile for spark are correct.</p>
+ </li>
+ <li>
+ <p>Use the following command :</p>
+ </li>
+</ol>
+<pre><code>```
+ "mvn -Pspark-2.1 -Dspark.version {yourSparkVersion} clean package"
+```
+
+Note : Refrain from using "mvn clean package" without specifying the profile.
+</code></pre>
+<h2 id="query-failure-with-generic-error-on-the-beeline">Query Failure with Generic Error
+ on the Beeline</h2>
+<p><strong>Symptom</strong></p>
+<p>Query fails on the executor side and generic error message is printed on the beeline console</p>
+<p><img src="../../../webapp/docs/latest/images/query_failure_beeline.png?raw=true"
+ alt="Query Failure Beeline"></p>
+<p><strong>Possible Causes</strong></p>
+<ul>
+ <li>In Query flow, Table B-Tree will be loaded into memory on the driver side and filter
+ condition is validated against the min-max of each block to identify false positive,<br>
+ Once the blocks are selected, based on number of available executors, blocks will be
+ distributed to each executor node as shown in below driver logs snapshot
+ </li>
+</ul>
+<p><img src="../../../webapp/docs/latest/images/query_failure_logs.png?raw=true"
+ alt="Query Failure Logs"></p>
+<ul>
+ <li>
+ <p>When the error occurs in driver side while b-tree loading or block distribution, detail
+ error message will be printed on the beeline console and error trace will be printed on
+ the driver logs.</p>
+ </li>
+ <li>
+ <p>When the error occurs in the executor side, generic error message will be printed as
+ shown in issue description.</p>
+ </li>
+</ul>
+<p><img src="../../../webapp/docs/latest/images/query_failure_job_details.png?raw=true"
+ alt="Query Failure Job Details"></p>
+<ul>
+ <li>Details of the failed stages can be seen in the Spark Application UI by clicking on the
+ failed stages on the failed job as shown in previous snapshot
+ </li>
+</ul>
+<p><img src="../../../webapp/docs/latest/images/query_failure_spark_ui.png?raw=true"
+ alt="Query Failure Spark UI"></p>
+<p><strong>Procedure</strong></p>
+<p>Details of the error can be analyzed in details using executor logs available in stdout</p>
+<p><img src="../../../webapp/docs/latest/images/query_failure_procedure.png?raw=true"
+ alt="Query Failure Spark UI"></p>
+<p>Below snapshot shows executor logs with error message for query failure which can be helpful to
+ locate the error</p>
+<p><img src="../../../webapp/docs/latest/images/query_failure_issue.png?raw=true"
+ alt="Query Failure Spark UI"></p>
+<h2 id="failed-to-execute-load-query-on-cluster">Failed to execute load query on cluster.
+</h2>
+<p><strong>Symptom</strong></p>
+<p>Load query failed with the following exception:</p>
+<pre><code>Dictionary file is locked for updation.
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The carbon.properties file is not identical in all the nodes of the cluster.</p>
+<p><strong>Procedure</strong></p>
+<p>Follow the steps to ensure the carbon.properties file is consistent across all the nodes:</p>
<ol>
- <li><p>Copy the carbon.properties file from the master node to all the other nodes in the
- cluster. For example, you can use ssh to copy this file to all the nodes.</p></li>
- <li><p>For the changes to take effect, restart the Spark cluster.</p></li>
-</ol><h2>Failed to execute insert query on cluster.</h2><p><strong>Symptom</strong></p><p>Load query
- failed with the following exception:</p><p><code>
- Dictionary file is locked for updation.
-</code></p><p><strong>Possible Cause</strong></p><p>The carbon.properties file is not identical in
- all the nodes of the cluster.</p><p><strong>Procedure</strong></p><p>Follow the steps to ensure
- the carbon.properties file is consistent across all the nodes:</p>
+ <li>
+ <p>Copy the carbon.properties file from the master node to all the other nodes in the
+ cluster.<br>
+ For example, you can use ssh to copy this file to all the nodes.</p>
+ </li>
+ <li>
+ <p>For the changes to take effect, restart the Spark cluster.</p>
+ </li>
+</ol>
+<h2 id="failed-to-execute-insert-query-on-cluster">Failed to execute insert query on
+ cluster.</h2>
+<p><strong>Symptom</strong></p>
+<p>Load query failed with the following exception:</p>
+<pre><code>Dictionary file is locked for updation.
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The carbon.properties file is not identical in all the nodes of the cluster.</p>
+<p><strong>Procedure</strong></p>
+<p>Follow the steps to ensure the carbon.properties file is consistent across all the nodes:</p>
<ol>
- <li><p>Copy the carbon.properties file from the master node to all the other nodes in the
- cluster. For example, you can use scp to copy this file to all the nodes.</p></li>
- <li><p>For the changes to take effect, restart the Spark cluster.</p></li>
-</ol><h2>Failed to connect to hiveuser with thrift</h2><p><strong>Symptom</strong></p><p>We get the
- following exception :</p><p><code>
- Cannot connect to hiveuser.
-</code></p><p><strong>Possible Cause</strong></p><p>The external process does not have permission to
- access.</p><p><strong>Procedure</strong></p><p>Ensure that the Hiveuser in mysql must allow its
- access to the external processes.</p><h2>Failure to read the metastore db during table
- creation.</h2><p><strong>Symptom</strong></p><p>We get the following exception on trying to
- connect :</p><p><code>
- Cannot read the metastore db
-</code></p><p><strong>Possible Cause</strong></p><p>The metastore db is dysfunctional.</p><p>
- <strong>Procedure</strong></p><p>Remove the metastore db from the carbon.metastore in the Spark
- Directory.</p><h2>Failed to load data on the cluster</h2><p><strong>Symptom</strong></p><p>Data
- loading fails with the following exception :</p><p><code>
- Data Load failure exeception
-</code></p><p><strong>Possible Cause</strong></p><p>The following issue can cause the failure :</p>
+ <li>
+ <p>Copy the carbon.properties file from the master node to all the other nodes in the
+ cluster.<br>
+ For example, you can use scp to copy this file to all the nodes.</p>
+ </li>
+ <li>
+ <p>For the changes to take effect, restart the Spark cluster.</p>
+ </li>
+</ol>
+<h2 id="failed-to-connect-to-hiveuser-with-thrift">Failed to connect to hiveuser with
+ thrift</h2>
+<p><strong>Symptom</strong></p>
+<p>We get the following exception :</p>
+<pre><code>Cannot connect to hiveuser.
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The external process does not have permission to access.</p>
+<p><strong>Procedure</strong></p>
+<p>Ensure that the Hiveuser in mysql must allow its access to the external processes.</p>
+<h2 id="failure-to-read-the-metastore-db-during-table-creation">Failure to read the
+ metastore db during table creation.</h2>
+<p><strong>Symptom</strong></p>
+<p>We get the following exception on trying to connect :</p>
+<pre><code>Cannot read the metastore db
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The metastore db is dysfunctional.</p>
+<p><strong>Procedure</strong></p>
+<p>Remove the metastore db from the carbon.metastore in the Spark Directory.</p>
+<h2 id="failed-to-load-data-on-the-cluster">Failed to load data on the cluster</h2>
+<p><strong>Symptom</strong></p>
+<p>Data loading fails with the following exception :</p>
+<pre><code>Data Load failure exeception
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The following issue can cause the failure :</p>
<ol>
- <li><p>The core-site.xml, hive-site.xml, yarn-site and carbon.properties are not consistent
- across all nodes of the cluster.</p></li>
- <li><p>Path to hdfs ddl is not configured correctly in the carbon.properties.</p></li>
-</ol><p><strong>Procedure</strong></p><p>Follow the steps to ensure the following configuration
- files are consistent across all the nodes:</p>
+ <li>
+ <p>The core-site.xml, hive-site.xml, yarn-site and carbon.properties are not consistent
+ across all nodes of the cluster.</p>
+ </li>
+ <li>
+ <p>Path to hdfs ddl is not configured correctly in the carbon.properties.</p>
+ </li>
+</ol>
+<p><strong>Procedure</strong></p>
+<p>Follow the steps to ensure the following configuration files are consistent across all the
+ nodes:</p>
<ol>
- <li><p>Copy the core-site.xml, hive-site.xml, yarn-site,carbon.properties files from the master
- node to all the other nodes in the cluster. For example, you can use scp to copy this file
- to all the nodes.</p>
- <p>Note : Set the path to hdfs ddl in carbon.properties in the master node.</p></li>
- <li><p>For the changes to take effect, restart the Spark cluster.</p></li>
-</ol><h2>Failed to insert data on the cluster</h2><p><strong>Symptom</strong></p><p>Insertion fails
- with the following exception :</p><p><code>
- Data Load failure exeception
-</code></p><p><strong>Possible Cause</strong></p><p>The following issue can cause the failure :</p>
+ <li>
+ <p>Copy the core-site.xml, hive-site.xml, yarn-site,carbon.properties files from the master
+ node to all the other nodes in the cluster.<br>
+ For example, you can use scp to copy this file to all the nodes.</p>
+ <p>Note : Set the path to hdfs ddl in carbon.properties in the master node.</p>
+ </li>
+ <li>
+ <p>For the changes to take effect, restart the Spark cluster.</p>
+ </li>
+</ol>
+<h2 id="failed-to-insert-data-on-the-cluster">Failed to insert data on the cluster</h2>
+<p><strong>Symptom</strong></p>
+<p>Insertion fails with the following exception :</p>
+<pre><code>Data Load failure exeception
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>The following issue can cause the failure :</p>
<ol>
- <li><p>The core-site.xml, hive-site.xml, yarn-site and carbon.properties are not consistent
- across all nodes of the cluster.</p></li>
- <li><p>Path to hdfs ddl is not configured correctly in the carbon.properties.</p></li>
-</ol><p><strong>Procedure</strong></p><p>Follow the steps to ensure the following configuration
- files are consistent across all the nodes:</p>
+ <li>
+ <p>The core-site.xml, hive-site.xml, yarn-site and carbon.properties are not consistent
+ across all nodes of the cluster.</p>
+ </li>
+ <li>
+ <p>Path to hdfs ddl is not configured correctly in the carbon.properties.</p>
+ </li>
+</ol>
+<p><strong>Procedure</strong></p>
+<p>Follow the steps to ensure the following configuration files are consistent across all the
+ nodes:</p>
<ol>
- <li><p>Copy the core-site.xml, hive-site.xml, yarn-site,carbon.properties files from the master
- node to all the other nodes in the cluster. For example, you can use scp to copy this file
- to all the nodes.</p>
- <p>Note : Set the path to hdfs ddl in carbon.properties in the master node.</p></li>
- <li><p>For the changes to take effect, restart the Spark cluster.</p></li>
-</ol><h2>Failed to execute Concurrent Operations(Load,Insert,Update) on table by multiple
- workers.</h2><p><strong>Symptom</strong></p><p>Execution fails with the following exception
- :</p><p><code>
- Table is locked for updation.
-</code></p><p><strong>Possible Cause</strong></p><p>Concurrency not supported.</p><p><strong>Procedure</strong>
-</p><p>Worker must wait for the query execution to complete and the table to release the lock for
- another query execution to succeed..</p><h2>Failed to create a table with a single numeric
- column.</h2><p><strong>Symptom</strong></p><p>Execution fails with the following exception :</p>
-<p><code>
- Table creation fails.
-</code></p><p><strong>Possible Cause</strong></p><p>Behavior not supported.</p><p>
- <strong>Procedure</strong></p><p>A single column that can be considered as dimension is
- mandatory for table creation.</p>
\ No newline at end of file
+ <li>
+ <p>Copy the core-site.xml, hive-site.xml, yarn-site,carbon.properties files from the master
+ node to all the other nodes in the cluster.<br>
+ For example, you can use scp to copy this file to all the nodes.</p>
+ <p>Note : Set the path to hdfs ddl in carbon.properties in the master node.</p>
+ </li>
+ <li>
+ <p>For the changes to take effect, restart the Spark cluster.</p>
+ </li>
+</ol>
+<h2 id="failed-to-execute-concurrent-operations">Failed to execute Concurrent Operations.
+</h2>
+<p><strong>Symptom</strong></p>
+<p>Execution of Concurrent Operations (Load,Insert,Update) on table by multiple workers fails with
+ the following exception :</p>
+<pre><code>Table is locked for updation.
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>Concurrency not supported.</p>
+<p><strong>Procedure</strong></p>
+<p>Worker must wait for the query execution to complete and the table to release the lock for
+ another query execution to succeed\u2026</p>
+<h2 id="failed-to-create-a-table-with-a-single-numeric-column">Failed to create a table
+ with a single numeric column.</h2>
+<p><strong>Symptom</strong></p>
+<p>Execution fails with the following exception :</p>
+<pre><code>Table creation fails.
+</code></pre>
+<p><strong>Possible Cause</strong></p>
+<p>Behavior not supported.</p>
+<p><strong>Procedure</strong></p>
+<p>A single column that can be considered as dimension is mandatory for table creation.</p>
+<h2 id="data-failure-because-of-bad-records">Data Failure because of Bad Records</h2>
+<p><strong>Symptom</strong></p>
+<p>Data Loading fails with the following exception</p>
+<pre><code>Error: java.lang.Exception: Data load failed due to Bad record
+</code></pre>
+<p><strong>Possible Causes</strong></p>
+<p>The parameter BAD_RECORDS_ACTION has not been specified in the Query.</p>
+<p><strong>Procedure</strong></p>
+<p>Set the following parameter in the load command OPTIONS as shown below :</p>
+<p>\u2018BAD_RECORDS_ACTION\u2019='FORCE\u2018</p>
+<p><em>Example :</em></p>
+<pre><code>LOAD DATA INPATH 'hdfs://hacluster/user/loader/moredata01.csv' INTO TABLE flow_carbon_256b OPTIONS('DELIMITER'=',', 'BAD_RECORDS_ACTION'='FORCE');
+</code></pre>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/main/webapp/index.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/index.html b/src/main/webapp/index.html
index 9595207..6453275 100644
--- a/src/main/webapp/index.html
+++ b/src/main/webapp/index.html
@@ -80,12 +80,13 @@
class="caret"></span></a>
<ul class="dropdown-menu">
<li>
- <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Contributing+to+CarbonData"
+ <a href="https://github.com/apache/incubator-carbondata/blob/master/docs/How-to-contribute-to-Apache-CarbonData.md"
target="_blank">Contributing to CarbonData</a></li>
<li>
<a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Committers"
target="_blank">Project Committers</a></li>
<li><a href="meetup.html">CarbonData Meetups </a></li>
+ <li><a href="security.html">Apache CarbonData Security</a></li>
</ul>
</li>
<li class="dropdown">
@@ -119,9 +120,29 @@
target="_blank">Thanks</a></li>
</ul>
</li>
+ <li>
+ <a href="#" id="search-icon" ><i class="fa fa-search" aria-hidden="true"></i></a>
+
+ </li>
</ul>
</div><!--/.nav-collapse -->
+ <div id="search-box" >
+ <form method="get" action="http://www.google.com/search">
+ <div class="search-block">
+ <table border="0" cellpadding="0" width="100%">
+ <tr>
+ <td style="width:80%">
+ <input type="text" name="q" size=" 5" maxlength="255" value="" class="search-input" />
+ </td>
+ <td style="width:20%">
+ <input type="submit" value="Search" /></td></tr>
+ <tr><td align="left" style="font-size:75%" colspan="2">
+ <input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked /> Only search for CarbonData </td></tr>
+ </table>
+ </div>
+ </form>
+ </div>
</div>
</nav>
</header> <!-- end Header part -->
@@ -360,7 +381,8 @@
</ul>
<p class="social-icons">
- <a> <i class="fa fa-facebook-square" aria-hidden="true"></i></a>
+ <a href="https://www.facebook.com/carbondata/" target="_blank">
+ <i class="fa fa-facebook-square" aria-hidden="true"></i></a>
<a href="https://twitter.com/search?q=%23CarbonData" target="_blank"> <i
class="fa fa-twitter-square" aria-hidden="true"></i></a>
<a> <i class="fa fa-linkedin-square" aria-hidden="true"></i></a>
@@ -524,7 +546,6 @@
<img src="images/CDincubator.png" alt=""/>
</div>
</div>
-
<div class="col-md-9 col-sm-8">
<div class="cd-apachetxtblock">
<h4 class="title">Apache & OpenSource
@@ -574,7 +595,7 @@
<div class="bottomcontant">
<div class="container-fluid">
<div class="col-md-8 col-sm-8">
- <p class="copyright-txt">Copyright � 2017. All rights reserved |
+ <p class="copyright-txt">Copyright � 2016-2017 Apache CarbonData. All rights reserved |
<a href="http://www.apache.org/" class="term-links">Apache Software
Foundation </a> |
<a href="http://www.apache.org/foundation/policies/privacy.html"
@@ -591,7 +612,6 @@
</div>
</div>
</div>
-
</footer>
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/main/webapp/js/custom.js
----------------------------------------------------------------------
diff --git a/src/main/webapp/js/custom.js b/src/main/webapp/js/custom.js
index 81b82ba..72a873c 100644
--- a/src/main/webapp/js/custom.js
+++ b/src/main/webapp/js/custom.js
@@ -63,5 +63,12 @@ $(document).ready(function(){
$("#viewpage").load(pagelink);
});
-
+
+ /*Search box*/
+
+ $(function() {
+ $( "#search-icon" ).click(function() {
+ $( "#search-box" ).toggle();
+ });
+ });
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/main/webapp/meetup.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/meetup.html b/src/main/webapp/meetup.html
index 8e6c74a..b3d1776 100644
--- a/src/main/webapp/meetup.html
+++ b/src/main/webapp/meetup.html
@@ -1,6 +1,6 @@
<!DOCTYPE html>
<html lang="en">
- <head>
+<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
@@ -9,163 +9,233 @@
<title>CarbonData:Meetup</title>
<!-- Bootstrap -->
-
+
<link rel="stylesheet" href="css/bootstrap.min.css">
- <link href="css/style.css" rel="stylesheet">
- <link href="css/print.css" rel="stylesheet" >
+ <link href="css/style.css" rel="stylesheet">
+ <link href="css/print.css" rel="stylesheet">
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
<!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
<!--[if lt IE 9]>
- <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
- <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
- </head>
- <body>
- <header>
- <nav class="navbar navbar-default navbar-custom cd-navbar-wrapper" >
- <div class="container">
- <div class="navbar-header">
- <button aria-controls="navbar" aria-expanded="false" data-target="#navbar" data-toggle="collapse" class="navbar-toggle collapsed" type="button">
- <span class="sr-only">Toggle navigation</span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- </button>
- <a href="index.html" class="logo">
- <img src="images/CarbonDataLogo.png" alt="CarbonData logo" title="CarbocnData logo" />
- </a>
+</head>
+<body>
+<header>
+ <nav class="navbar navbar-default navbar-custom cd-navbar-wrapper">
+ <div class="container">
+ <div class="navbar-header">
+ <button aria-controls="navbar" aria-expanded="false" data-target="#navbar"
+ data-toggle="collapse" class="navbar-toggle collapsed" type="button">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a href="index.html" class="logo">
+ <img src="images/CarbonDataLogo.png" alt="CarbonData logo"
+ title="CarbocnData logo"/>
+ </a>
+ </div>
+ <div class="navbar-collapse collapse cd_navcontnt" id="navbar">
+ <ul class="nav navbar-nav navbar-right navlist-custom">
+ <li><a href="index.html" class="hidden-xs"><i class="fa fa-home"
+ aria-hidden="true"></i> </a></li>
+ <li><a href="index.html" class="hidden-lg hidden-md hidden-sm">Home</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button"
+ aria-haspopup="true" aria-expanded="false">Download <span
+ class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/1.0.0-incubating"
+ target="_blank">Apache CarbonData 1.0.0</a></li>
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.2.0-incubating"
+ target="_blank">Apache CarbonData 0.2.0</a></li>
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.1.1-incubating"
+ target="_blank">Apache CarbonData 0.1.1</a></li>
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.1.0-incubating"
+ target="_blank">Apache CarbonData 0.1.0</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
+ target="_blank">Release Archive</a></li>
+ </ul>
+ </li>
+ <li><a href="docs/latest/mainpage.html?page=userguide"
+ class="">Documentation</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button"
+ aria-haspopup="true" aria-expanded="false">Community <span
+ class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://github.com/apache/incubator-carbondata/blob/master/docs/How-to-contribute-to-Apache-CarbonData.md"
+ target="_blank">Contributing to CarbonData</a></li>
+ </li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Committers"
+ target="_blank">Project Committers</a></li>
+ <li><a href="meetup.html">CarbonData Meetups </a></li>
+ <li><a href="security.html">Apache CarbonData Security</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="http://www.apache.org/"
+ class="apache_link hidden-xs dropdown-toggle" data-toggle="dropdown"
+ role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a>
+ </li>
+ <li><a href="http://www.apache.org/licenses/"
+ target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html"
+ target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="http://www.apache.org/"
+ class="hidden-lg hidden-md hidden-sm dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a>
+ </li>
+ <li><a href="http://www.apache.org/licenses/"
+ target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html"
+ target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+ <li>
+ <a href="#" id="search-icon"><i class="fa fa-search" aria-hidden="true"></i></a>
+
+ </li>
+ </ul>
+ </div><!--/.nav-collapse -->
+ <div id="search-box">
+ <form method="get" action="http://www.google.com/search">
+ <div class="search-block">
+ <table border="0" cellpadding="0" width="100%">
+ <tr>
+ <td style="width:80%">
+ <input type="text" name="q" size=" 5" maxlength="255" value=""
+ class="search-input"/>
+ </td>
+ <td style="width:20%">
+ <input type="submit" value="Search"/></td>
+ </tr>
+ <tr>
+ <td align="left" style="font-size:75%" colspan="2">
+ <input type="checkbox" name="sitesearch"
+ value="carbondata.apache.org" checked/> Only search for
+ CarbonData
+ </td>
+ </tr>
+ </table>
+ </div>
+ </form>
+ </div>
</div>
- <div class="navbar-collapse collapse cd_navcontnt" id="navbar">
- <ul class="nav navbar-nav navbar-right navlist-custom">
- <li><a href="index.html" class="hidden-xs"><i class="fa fa-home" aria-hidden="true"></i> </a></li>
- <li><a href="index.html" class="hidden-lg hidden-md hidden-sm">Home</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Download <span class="caret"></span></a>
- <ul class="dropdown-menu">
- <li>
- <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/1.0.0-incubating"
- target="_blank">Apache CarbonData 1.0.0</a></li>
- <li>
- <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.2.0-incubating"
- target="_blank">Apache CarbonData 0.2.0</a></li>
- <li>
- <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.1.1-incubating"
- target="_blank">Apache CarbonData 0.1.1</a></li>
- <li>
- <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.1.0-incubating"
- target="_blank">Apache CarbonData 0.1.0</a></li>
- <li>
- <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
- target="_blank">Release Archive</a></li>
- </ul>
- </li>
- <li><a href="docs/latest/mainpage.html?page=userguide" class="">Documentation</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Community <span class="caret"></span></a>
- <ul class="dropdown-menu">
- <li><a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Contributing+to+CarbonData" target="_blank">Contributing to CarbonData</a></li>
- <li><a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Committers" target="_blank">Project Committers</a></li>
- <li><a href="meetup.html">CarbonData Meetups </a></li>
- </ul>
- </li>
- <li class="dropdown">
- <a href="http://www.apache.org/" class="apache_link hidden-xs dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
- <ul class="dropdown-menu">
- <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
- <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
- <li><a href="http://www.apache.org/foundation/sponsorship.html" target="_blank">Sponsorship</a></li>
- <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
- </ul>
- </li>
- <li class="dropdown">
- <a href="http://www.apache.org/" class="hidden-lg hidden-md hidden-sm dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
- <ul class="dropdown-menu">
- <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
- <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
- <li><a href="http://www.apache.org/foundation/sponsorship.html" target="_blank">Sponsorship</a></li>
- <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
- </ul>
- </li>
- </ul>
- </div><!--/.nav-collapse -->
- </div>
</nav>
- </header> <!-- end Header part -->
-
- <div class="fixed-padding"></div> <!-- top padding with fixde header -->
-
- <section style="position:relative; overflow:hidden ">
+</header> <!-- end Header part -->
+
+<div class="fixed-padding"></div> <!-- top padding with fixde header -->
+
+<section style="position:relative; overflow:hidden ">
<div class="meetup-banber-bg">
- <img src="images/CDbannerSliderbg.jpg" alt="CarbonData banner" />
+ <img src="images/CDbannerSliderbg.jpg" alt="CarbonData banner"/>
</div>
- <div class="meetup-banber">
- <div class="container-fluid">
- <div class="col-md-12 col-sm-12">
- <h1 class="title-heading">ComingUp Meetups
- <span class="title-underline"></span>
- </h1>
- <p class="text-center">
- <span class="txt"> Venue </span>
- <span class="txt"> To be Scheduled </span>
- </p>
-
- <!-- <p class="text-center">
- <a href="#" class="cominguplink"> </a>
- </p>-->
+ <div class="meetup-banber">
+ <div class="container-fluid">
+ <div class="col-md-12 col-sm-12">
+ <h1 class="title-heading">Meetups
+ <span class="title-underline"></span>
+ </h1>
</div>
- </div>
- </div>
- </section>
+ </div>
+ </div>
+</section>
- <section>
- <div class="container-fluid">
- <div class="col-md-12 col-sm-12">
- <div class="meetup-details">
+<section>
+ <div class="container-fluid">
+ <div class="col-md-12 col-sm-12">
+ <div class="meetup-details">
<h2 class="title-heading"> Premiure Events
- <span class="title-underline"></span>
+ <span class="title-underline"></span>
</h2>
<div class="meetup-info">
- <div class="row">
- <div class="col-md-10 col-sm-9">
- <h2 class="meetup-title">Oscon - Austin TX<span class="meetup-date">May 2016</span></h2>
- <!--<p class="meetup-location"> </p>-->
- <p>The agenda for the meetup was 'Carbondata : A New Hadoop File Format for Faster Data Analysis'. It focused on the inception and introduction of CarbonData to the open source community.</p>
- </div>
- <div class="col-md-2 col-sm-3">
- <div class="meetup-icon">
- <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=66850609" target="blank">
- <i class="fa fa-file-pdf-o" aria-hidden="true"></i>
- </a>
-
+ <div class="row">
+ <div class="col-md-10 col-sm-9">
+ <h2 class="meetup-title">Oscon - Austin TX<span class="meetup-date">May 2016</span>
+ </h2>
+ <!--<p class="meetup-location"> </p>-->
+ <p>The agenda for the meetup was 'Carbondata : A New Hadoop File Format
+ for Faster Data Analysis'. It focused on the inception and
+ introduction of CarbonData to the open source community.</p>
+ </div>
+ <div class="col-md-2 col-sm-3">
+ <div class="meetup-icon">
+ <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=66850609"
+ target="blank">
+ <i class="fa fa-angle-right" aria-hidden="true"></i>
+ </a>
+ </div>
+ </div>
</div>
- </div>
- </div>
</div>
<div class="meetup-info">
- <div class="row">
- <div class="col-md-10 col-sm-9">
- <h2 class="meetup-title">Beijing<span class="meetup-date">Oct 2016</span></h2>
- <!-- <p class="meetup-location"> </p>-->
- <p>The agenda for the meetup was 'CarbonData : An Indexed Columnar File Format for Interactive query'. It detailedly introduced the architecture and techniques of Apache CarbonData ,followed by a tutorial session and open discussion about future plans of Apache Carbondata.</p>
- </div>
- <div class="col-md-2 col-sm-3">
- <div class="meetup-icon">
- <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/10.29+Beijing+Apache+CarbonData+Meetup+PPT" target="blank">
- <i class="fa fa-file-pdf-o" aria-hidden="true"></i>
- </a>
-
-
-
+ <div class="row">
+ <div class="col-md-10 col-sm-9">
+ <h2 class="meetup-title">Beijing<span
+ class="meetup-date">Oct 29, 2016</span></h2>
+ <!-- <p class="meetup-location"> </p>-->
+ <p>The agenda for the meetup was 'CarbonData : An Indexed Columnar File
+ Format for Interactive query'. It detailedly introduced the
+ architecture and techniques of Apache CarbonData ,followed by a
+ tutorial session and open discussion about future plans of Apache
+ Carbondata.</p>
</div>
- </div>
- </div>
+ <div class="col-md-2 col-sm-3">
+ <div class="meetup-icon">
+ <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=66849573"
+ target="blank">
+ <i class="fa fa-angle-right" aria-hidden="true"></i>
+ </a>
+ </div>
+ </div>
+ </div>
+ </div>
+ <div class="meetup-info">
+ <div class="row">
+ <div class="col-md-10 col-sm-9">
+ <h2 class="meetup-title">Spark Summit East 2017-Apache CarbonData<span class="meetup-date">Feb 8, 2017</span>
+ </h2>
+ <!--<p class="meetup-location"> </p>-->
+ <p>The agenda for the meetup was 'Apache Carbondata: An Indexed Columnar File Format for Interactive Query with Spark SQL'</p>
+ </div>
+ <div class="col-md-2 col-sm-3">
+ <div class="meetup-icon">
+ <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=68714802"
+ target="blank">
+ <i class="fa fa-angle-right" aria-hidden="true"></i>
+ </a>
+ </div>
+ </div>
+ </div>
</div>
- </div>
</div>
- </div>
- </section>
-
+ </div>
+ </div>
+</section>
+
<!--
<footer>
<div class="topcontant">
@@ -205,11 +275,11 @@
</div>
</footer> -->
- <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+<!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
- <script src="js/jquery.min.js"></script>
- <!-- Include all compiled plugins (below), or include individual files as needed -->
- <script src="js/bootstrap.min.js"></script>
- <script src="js/custom.js"></script>
- </body>
- </html>
\ No newline at end of file
+<script src="js/jquery.min.js"></script>
+<!-- Include all compiled plugins (below), or include individual files as needed -->
+<script src="js/bootstrap.min.js"></script>
+<script src="js/custom.js"></script>
+</body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/main/webapp/security.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/security.html b/src/main/webapp/security.html
new file mode 100644
index 0000000..205eba4
--- /dev/null
+++ b/src/main/webapp/security.html
@@ -0,0 +1,173 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href='images/favicon.ico' rel='shortcut icon' type='image/x-icon'>
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>CarbonData:Security</title>
+
+ <!-- Bootstrap -->
+
+ <link rel="stylesheet" href="css/bootstrap.min.css">
+ <link href="css/style.css" rel="stylesheet">
+ <link href="css/print.css" rel="stylesheet" >
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+</head>
+<body>
+<header>
+ <nav class="navbar navbar-default navbar-custom cd-navbar-wrapper" >
+ <div class="container">
+ <div class="navbar-header">
+ <button aria-controls="navbar" aria-expanded="false" data-target="#navbar" data-toggle="collapse" class="navbar-toggle collapsed" type="button">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a href="index.html" class="logo">
+ <img src="images/CarbonDataLogo.png" alt="CarbonData logo" title="CarbocnData logo" />
+ </a>
+ </div>
+ <div class="navbar-collapse collapse cd_navcontnt" id="navbar">
+ <ul class="nav navbar-nav navbar-right navlist-custom">
+ <li><a href="index.html" class="hidden-xs"><i class="fa fa-home" aria-hidden="true"></i> </a></li>
+ <li><a href="index.html" class="hidden-lg hidden-md hidden-sm">Home</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Download <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/1.0.0-incubating"
+ target="_blank">Apache CarbonData 1.0.0</a></li>
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.2.0-incubating"
+ target="_blank">Apache CarbonData 0.2.0</a></li>
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.1.1-incubating"
+ target="_blank">Apache CarbonData 0.1.1</a></li>
+ <li>
+ <a href="https://www.apache.org/dyn/closer.lua/incubator/carbondata/0.1.0-incubating"
+ target="_blank">Apache CarbonData 0.1.0</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
+ target="_blank">Release Archive</a></li>
+ </ul>
+ </li>
+ <li><a href="docs/latest/mainpage.html?page=userguide" class="">Documentation</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Community <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://github.com/apache/incubator-carbondata/blob/master/docs/How-to-contribute-to-Apache-CarbonData.md"
+ target="_blank">Contributing to CarbonData</a></li></li>
+ <li><a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Committers" target="_blank">Project Committers</a></li>
+ <li><a href="meetup.html">CarbonData Meetups </a></li>
+ <li><a href="security.html">Apache CarbonData Security </a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="apache_link hidden-xs dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html" target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="hidden-lg hidden-md hidden-sm dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html" target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+ <li>
+ <a href="#" id="search-icon" ><i class="fa fa-search" aria-hidden="true"></i></a>
+ </li>
+ </ul>
+ </div><!--/.nav-collapse -->
+
+ <div id="search-box" >
+ <form method="get" action="http://www.google.com/search">
+ <div class="search-block">
+ <table border="0" cellpadding="0" width="100%">
+ <tr>
+ <td style="width:80%">
+ <input type="text" name="q" size=" 5" maxlength="255" value="" class="search-input" />
+ </td>
+ <td style="width:20%">
+ <input type="submit" value="Search" /></td></tr>
+ <tr><td align="left" style="font-size:75%" colspan="2">
+ <input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked /> Only search for CarbonData </td></tr>
+ </table>
+ </div>
+ </form>
+ </div>
+
+ </div>
+ </nav>
+</header> <!-- end Header part -->
+
+<div class="fixed-padding"></div> <!-- top padding with fixde header -->
+
+<section style="position:relative; overflow:hidden ">
+ <div class="meetup-banber-bg">
+ <img src="images/CDbannerSliderbg.jpg" alt="CarbonData banner" />
+ </div>
+
+ <div class="meetup-banber" style="padding:30px">
+ <div class="container-fluid">
+ <div class="col-md-12 col-sm-12">
+ <h1 class="title-heading">
+ Indexed Columnar Data Format for Faster Analytics on <br>Big Data Platforms.
+
+ <span class="title-underline"></span>
+ </h1>
+ </div>
+ </div>
+ </div>
+</section>
+
+<section>
+ <div class="container">
+ <div class="row">
+ <div class="col-md-12 col-sm-12">
+ <div class="security-block">
+ <h1 class="title-heading"> Apache CarbonData Security
+ <span class="title-underline"></span>
+ </h1>
+
+ <p>
+ The Apache Software Foundation takes a rigorous standpoint in annihilating the security issues in its software projects. Apache CarbonData is highly sensitive and forthcoming to issues pertaining to its features and functionality.
+ </p>
+ <p>If you have apprehensions regarding CarbonData\u2019s security or you discover vulnerability or potential threat, don\u2019t hesitate to get in touch with the Apache Security Team by dropping a mail at <a href="mailto:security@apache.org"> security@apache.org</a>. In the mail, specify the project name CarbonData with the description of the issue or potential threat. You are also urged to recommend the way to reproduce and replicate the issue. The security team and the CarbonData community will get back to you after assessing and analysing the findings.</p>
+ <p>
+ PLEASE PAY ATTENTION to report the security issue on the security email before disclosing it on public domain.
+ </p>
+ <p>
+ The ASF Security Team maintains a page with the description of how vulnerabilities and potential threats are handled, check their <a href="http://www.apache.org/security/" target="_blank">Web page</a> for more Details.
+ </p>
+
+ </div>
+ </div>
+ </div>
+ </div>
+</section>
+
+
+<!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+
+<script src="js/jquery.min.js"></script>
+<!-- Include all compiled plugins (below), or include individual files as needed -->
+<script src="js/bootstrap.min.js"></script>
+<script src="js/custom.js"></script>
+<script src="js/mdNavigation.js" type="text/javascript"></script>
+</body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/site/markdown/faq.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/faq.md b/src/site/markdown/faq.md
index c7956db..d02d296 100755
--- a/src/site/markdown/faq.md
+++ b/src/site/markdown/faq.md
@@ -18,32 +18,104 @@
-->
# FAQs
-* **Auto Compaction not Working**
- The Property carbon.enable.auto.load.merge in carbon.properties need to be set to true.
+* [Can we preserve Segments from Compaction?](#can-we-preserve-segments-from-compaction)
+* [Can we disable horizontal compaction?](#can-we-disable-horizontal-compaction)
+* [What is horizontal compaction?](#what-is-horizontal-compaction)
+* [How to enable Compaction while data loading?](#how-to-enable-compaction-while-data-loading)
+* [Where are Bad Records Stored in CarbonData?](#where-are-bad-records-stored-in-carbondata)
+* [What are Bad Records?](#what-are-bad-records)
+* [Can we use CarbonData on Standalone Spark Cluster?](#can-we-use-carbondata-on-standalone-spark-cluster)
+* [What versions of Apache Spark are Compatible with CarbonData?](#what-versions-of-apache-spark-are-compatible-with-carbondata)
+* [Can we Load Data from excel?](#can-we-load-data-from-excel)
+* [How to enable Single Pass Data Loading?](#how-to-enable-single-pass-data-loading)
+* [What is Single Pass Data Loading?](#what-is-single-pass-data-loading)
+* [How to specify the data loading format for CarbonData ?](#how-to-specify-the-data-loading-format-for-carbondata)
+* [How to resolve store location can\u2019t be found?](#how-to-resolve-store-location-can-not-be-found)
+* [What is carbon.lock.type?]()
+* [How to enable Auto Compaction?](#how-to-enable-auto-compaction)
+* [How to resolve Abstract Method Error?](#how-to-resolve-abstract-method-error)
+* [Getting Exception on Creating a View](#getting-exception-on-creating-a-view)
+* [Is CarbonData supported for Windows?](#is-carbondata-supported-for-windows)
-* **Getting Abstract method error**
+## Can we preserve Segments from Compaction?
+If you want to preserve number of segments from being compacted then you can set the property **carbon.numberof.preserve.segments** equal to the **value of number of segments to be preserved**.
- You need to specify the spark version while using Maven to build project.
+Note : *No segments are preserved by Default.*
-* **Getting NotImplementedException for subquery using IN and EXISTS**
+## Can we disable horizontal compaction?
+Yes, to disable horizontal compaction, set **carbon.horizontal.compaction.enable** to ``FALSE`` in carbon.properties file.
- Subquery with in and exists not supported in CarbonData.
-
-* **Getting Exceptions on creating a view**
-
- View not supported in CarbonData.
-
-* **How to verify if ColumnGroups have been created as desired.**
+## What is horizontal compaction?
+Compaction performed after Update and Delete operations is referred as Horizontal Compaction. After every DELETE and UPDATE operation, horizontal compaction may occur in case the delta (DELETE/ UPDATE) files becomes more than specified threshold.
- Try using desc table query.
-
-* **Did anyone try to run CarbonData on windows? Is it supported on Windows?**
-
- We may provide support for windows in future. You are welcome to contribute if you want to add the support :)
+By default the parameter **carbon.horizontal.compaction.enable** enabling the horizontal compaction is set to ``TRUE``.
-* **Can we execute Concurrent Operations(Load,Insert,Update) on table by multiple workers.**
-
- Concurrency is not supported in current release of CarbonData.
-
+## How to enable Compaction while data loading?
+To enable compaction while data loading, set **carbon.enable.auto.load.merge** to ``TRUE`` in carbon.properties file.
+
+## Where are Bad Records Stored in CarbonData?
+The bad records are stored at the location set in carbon.badRecords.location in carbon.properties file.
+By default **carbon.badRecords.location** specifies the following location ``/opt/Carbon/Spark/badrecords``.
+
+## What are Bad Records?
+Records that fail to get loaded into the CarbonData due to data type incompatibility are classified as Bad Records.
+
+## Can we use CarbonData on Standalone Spark Cluster?
+Yes, CarbonData can be used on a Standalone spark cluster. But using a standalone cluster has following limitations:
+- single node cluster cannot be scaled up
+- the maximum memory and the CPU computation power has a fixed limit
+- the number of processors are limited in a single node cluster
+
+To harness the actual speed of execution of CarbonData on petabytes of data, it is suggested to use a Multinode Cluster.
+
+## What versions of Apache Spark are Compatible with CarbonData?
+Currently **Spark 1.6.2** and **Spark 2.1** is compatible with CarbonData.
+
+## Can we Load Data from excel?
+Yes, the data can be loaded from excel provided the data is in CSV format.
+
+## How to enable Single Pass Data Loading?
+You need to set **SINGLE_PASS** to ``True`` and append it to ``OPTIONS`` Section in the query as demonstrated in the Load Query below :
+```
+LOAD DATA local inpath '/opt/rawdata/data.csv' INTO table carbontable
+OPTIONS('DELIMITER'=',', 'QUOTECHAR'='"','FILEHEADER'='empno,empname,designation','USE_KETTLE'='FALSE')
+```
+Refer to [DML-operations-in-CarbonData](https://github.com/PallaviSingh1992/incubator-carbondata/blob/6b4dd5f3dea8c93839a94c2d2c80ab7a799cf209/docs/dml-operation-on-carbondata.md) for more details and example.
+
+## What is Single Pass Data Loading?
+Single Pass Loading enables single job to finish data loading with dictionary generation on the fly. It enhances performance in the scenarios where the subsequent data loading after initial load involves fewer incremental updates on the dictionary.
+This option specifies whether to use single pass for loading data or not. By default this option is set to ``FALSE``.
+
+## How to specify the data loading format for CarbonData?
+Edit carbon.properties file. Modify the value of parameter **carbon.data.file.version**.
+Setting the parameter **carbon.data.file.version** to ``1`` will support data loading in ``old format(0.x version)`` and setting **carbon.data.file.version** to ``2`` will support data loading in ``new format(1.x onwards)`` only.
+By default the data loading is supported using the new format.
+
+## How to resolve store location can not be found?
+Try creating ``carbonsession`` with ``storepath`` specified in the following manner :
+```
+val carbon = SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession(<store_path>)
+```
+Example:
+```
+val carbon = SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession("hdfs://localhost:9000/carbon/store ")
+```
+
+## What is carbon.lock.type?
+This property configuration specifies the type of lock to be acquired during concurrent operations on table. This property can be set with the following values :
+- **LOCALLOCK** : This Lock is created on local file system as file. This lock is useful when only one spark driver (thrift server) runs on a machine and no other CarbonData spark application is launched concurrently.
+- **HDFSLOCK** : This Lock is created on HDFS file system as file. This lock is useful when multiple CarbonData spark applications are launched and no ZooKeeper is running on cluster and the HDFS supports, file based locking.
+
+## How to enable Auto Compaction?
+To enable compaction set **carbon.enable.auto.load.merge** to ``TRUE`` in the carbon.properties file.
+
+## How to resolve Abstract Method Error?
+You need to specify the ``spark version`` while using Maven to build project.
+
+## Getting Exception on Creating a View
+View not supported in CarbonData.
+
+## Is CarbonData supported for Windows?
+We may provide support for windows in future. You are welcome to contribute if you want to add the support :)
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/site/markdown/images/query_failure_beeline.png
----------------------------------------------------------------------
diff --git a/src/site/markdown/images/query_failure_beeline.png b/src/site/markdown/images/query_failure_beeline.png
new file mode 100644
index 0000000..e4ec22b
Binary files /dev/null and b/src/site/markdown/images/query_failure_beeline.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/site/markdown/images/query_failure_issue.png
----------------------------------------------------------------------
diff --git a/src/site/markdown/images/query_failure_issue.png b/src/site/markdown/images/query_failure_issue.png
new file mode 100644
index 0000000..87270d2
Binary files /dev/null and b/src/site/markdown/images/query_failure_issue.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/site/markdown/images/query_failure_job_details.png
----------------------------------------------------------------------
diff --git a/src/site/markdown/images/query_failure_job_details.png b/src/site/markdown/images/query_failure_job_details.png
new file mode 100644
index 0000000..26e607d
Binary files /dev/null and b/src/site/markdown/images/query_failure_job_details.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/site/markdown/images/query_failure_logs.png
----------------------------------------------------------------------
diff --git a/src/site/markdown/images/query_failure_logs.png b/src/site/markdown/images/query_failure_logs.png
new file mode 100644
index 0000000..8fbdfa6
Binary files /dev/null and b/src/site/markdown/images/query_failure_logs.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/site/markdown/images/query_failure_procedure.png
----------------------------------------------------------------------
diff --git a/src/site/markdown/images/query_failure_procedure.png b/src/site/markdown/images/query_failure_procedure.png
new file mode 100644
index 0000000..9d2c81f
Binary files /dev/null and b/src/site/markdown/images/query_failure_procedure.png differ
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/7655ecd6/src/site/markdown/images/query_failure_spark_ui.png
----------------------------------------------------------------------
diff --git a/src/site/markdown/images/query_failure_spark_ui.png b/src/site/markdown/images/query_failure_spark_ui.png
new file mode 100644
index 0000000..1802760
Binary files /dev/null and b/src/site/markdown/images/query_failure_spark_ui.png differ