You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@carbondata.apache.org by ch...@apache.org on 2018/06/02 04:01:47 UTC
carbondata-site git commit: update md documents
Repository: carbondata-site
Updated Branches:
refs/heads/asf-site 61436c8e4 -> 44fe8a42d
update md documents
Project: http://git-wip-us.apache.org/repos/asf/carbondata-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata-site/commit/44fe8a42
Tree: http://git-wip-us.apache.org/repos/asf/carbondata-site/tree/44fe8a42
Diff: http://git-wip-us.apache.org/repos/asf/carbondata-site/diff/44fe8a42
Branch: refs/heads/asf-site
Commit: 44fe8a42d2101a5ec0ae6924ac5d712b72debc94
Parents: 61436c8
Author: chenliang613 <ch...@huawei.com>
Authored: Sat Jun 2 12:01:37 2018 +0800
Committer: chenliang613 <ch...@huawei.com>
Committed: Sat Jun 2 12:01:37 2018 +0800
----------------------------------------------------------------------
content/WEB-INF/classes/MdFileHandler.class | Bin 6144 -> 6144 bytes
content/WEB-INF/classes/html/header.html | 3 ++
content/data-management-on-carbondata.html | 4 +-
content/index.html | 7 +++
content/sdk-guide.html | 42 ++++++++++++++----
content/timeseries-datamap-guide.html | 8 ++--
.../webapp/data-management-on-carbondata.html | 4 +-
src/main/webapp/sdk-guide.html | 42 ++++++++++++++----
src/main/webapp/timeseries-datamap-guide.html | 8 ++--
src/site/markdown/sdk-guide.md | 43 +++++++++++++++----
10 files changed, 125 insertions(+), 36 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/content/WEB-INF/classes/MdFileHandler.class
----------------------------------------------------------------------
diff --git a/content/WEB-INF/classes/MdFileHandler.class b/content/WEB-INF/classes/MdFileHandler.class
index f39c098..58088b3 100644
Binary files a/content/WEB-INF/classes/MdFileHandler.class and b/content/WEB-INF/classes/MdFileHandler.class differ
http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/content/WEB-INF/classes/html/header.html
----------------------------------------------------------------------
diff --git a/content/WEB-INF/classes/html/header.html b/content/WEB-INF/classes/html/header.html
index d2ebd58..470de01 100644
--- a/content/WEB-INF/classes/html/header.html
+++ b/content/WEB-INF/classes/html/header.html
@@ -51,6 +51,9 @@
aria-expanded="false"> Download <span class="caret"></span></a>
<ul class="dropdown-menu">
<li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
+ target="_blank">Apache CarbonData 1.4.0</a></li>
+ <li>
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
target="_blank">Apache CarbonData 1.3.1</a></li>
<li>
http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/content/data-management-on-carbondata.html
----------------------------------------------------------------------
diff --git a/content/data-management-on-carbondata.html b/content/data-management-on-carbondata.html
index a9c7926..c81b02c 100644
--- a/content/data-management-on-carbondata.html
+++ b/content/data-management-on-carbondata.html
@@ -195,11 +195,11 @@
<a id="create-table" class="anchor" href="#create-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CREATE TABLE</h2>
<p>This command can be used to create a CarbonData table by specifying the list of fields along with the table properties. You can also specify the location where the table needs to be stored.</p>
<pre><code>CREATE TABLE [IF NOT EXISTS] [db_name.]table_name[(col_name data_type , ...)]
-STORED BY 'carbondata'
+STORED AS carbondata
[TBLPROPERTIES (property_name=property_value, ...)]
[LOCATION 'path']
</code></pre>
-<p><strong>NOTE:</strong> CarbonData also supports "STORED AS carbondata". Find example code at <a href="https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonSessionExample.scala" target=_blank>CarbonSessionExample</a> in the CarbonData repo.</p>
+<p><strong>NOTE:</strong> CarbonData also supports "STORED AS carbondata" and "USING carbondata". Find example code at <a href="https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonSessionExample.scala" target=_blank>CarbonSessionExample</a> in the CarbonData repo.</p>
<h3>
<a id="usage-guidelines" class="anchor" href="#usage-guidelines" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h3>
<p>Following are the guidelines for TBLPROPERTIES, CarbonData's additional table options can be set via carbon.properties.</p>
http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/content/index.html
----------------------------------------------------------------------
diff --git a/content/index.html b/content/index.html
index c201fb2..c1ea996 100644
--- a/content/index.html
+++ b/content/index.html
@@ -328,6 +328,13 @@
</h4>
<div class="linkblock">
<div class="block-row">
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
+ target="_blank">Apache CarbonData 1.4.0</a>
+ <span class="release-date">Mar 2018</span>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Apache+CarbonData+1.4.0+Release"
+ class="whatsnew" target="_blank">what's new</a>
+ </div>
+ <div class="block-row">
<a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
target="_blank">Apache CarbonData 1.3.1</a>
<span class="release-date">Mar 2018</span>
http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/content/sdk-guide.html
----------------------------------------------------------------------
diff --git a/content/sdk-guide.html b/content/sdk-guide.html
index 08044d1..7128c6c 100644
--- a/content/sdk-guide.html
+++ b/content/sdk-guide.html
@@ -340,6 +340,14 @@ These SDK writer output contains just a carbondata and carbonindex files. No met
</tbody>
</table>
<h2>
+<a id="run-sql-on-files-directly" class="anchor" href="#run-sql-on-files-directly" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Run SQL on files directly</h2>
+<p>Instead of creating table and query it, you can also query that file directly with SQL.</p>
+<h3>
+<a id="example" class="anchor" href="#example" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Example</h3>
+<pre><code>SELECT * FROM carbonfile.`$Path`
+</code></pre>
+<p>Find example code at <a href="https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/scala/org/apache/carbondata/examples/DirectSQLExample.scala" target=_blank>DirectSQLExample</a> in the CarbonData repo.</p>
+<h2>
<a id="api-list" class="anchor" href="#api-list" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>API List</h2>
<h3>
<a id="class-orgapachecarbondatasdkfilecarbonwriterbuilder" class="anchor" href="#class-orgapachecarbondatasdkfilecarbonwriterbuilder" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Class org.apache.carbondata.sdk.file.CarbonWriterBuilder</h3>
@@ -578,17 +586,22 @@ External client can make use of this reader to read CarbonData files without Car
String path = "./testWriteFiles";
CarbonReader reader = CarbonReader
.builder(path, "_temp")
- .projection(new String[]{"name", "age"})
+ .projection(new String[]{"stringField", "shortField", "intField", "longField",
+ "doubleField", "boolField", "dateField", "timeField", "decimalField"})
.build();
// 2. Read data
+ long day = 24L * 3600 * 1000;
int i = 0;
while (reader.hasNext()) {
- Object[] row = (Object[]) reader.readNextRow();
- System.out.println(row[0] + "\t" + row[1]);
- i++;
+ Object[] row = (Object[]) reader.readNextRow();
+ System.out.println(String.format("%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t",
+ i, row[0], row[1], row[2], row[3], row[4], row[5],
+ new Date((day * ((int) row[6]))), new Timestamp((long) row[7] / 1000), row[8]
+ ));
+ i++;
}
-
+
// 3. Close this reader
reader.close();
</code></pre>
@@ -597,12 +610,25 @@ External client can make use of this reader to read CarbonData files without Car
<a id="api-list-1" class="anchor" href="#api-list-1" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>API List</h2>
<h3>
<a id="class-orgapachecarbondatasdkfilecarbonreader" class="anchor" href="#class-orgapachecarbondatasdkfilecarbonreader" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Class org.apache.carbondata.sdk.file.CarbonReader</h3>
-<pre><code> /**
- * Return a new CarbonReaderBuilder instance
- */
+<pre><code> /**
+ * Return a new {@link CarbonReaderBuilder} instance
+ *
+ * @param tablePath table store path
+ * @param tableName table name
+ * @return CarbonReaderBuilder object
+ */
public static CarbonReaderBuilder builder(String tablePath, String tableName);
</code></pre>
<pre><code> /**
+ * Return a new CarbonReaderBuilder instance
+ * Default value of table name is table + tablePath + time
+ *
+ * @param tablePath table path
+ * @return CarbonReaderBuilder object
+ */
+ public static CarbonReaderBuilder builder(String tablePath);
+</code></pre>
+<pre><code> /**
* Return true if has next row
*/
public boolean hasNext();
http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/content/timeseries-datamap-guide.html
----------------------------------------------------------------------
diff --git a/content/timeseries-datamap-guide.html b/content/timeseries-datamap-guide.html
index b40150c..2400ce0 100644
--- a/content/timeseries-datamap-guide.html
+++ b/content/timeseries-datamap-guide.html
@@ -179,14 +179,14 @@
<div><h1>
<a id="carbondata-timeseries-datamap" class="anchor" href="#carbondata-timeseries-datamap" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CarbonData Timeseries DataMap</h1>
<ul>
-<li><a href="#timeseries-datamap-intoduction-(alpha-feature-in-1.3.0)">Timeseries DataMap</a></li>
+<li><a href="#timeseries-datamap-intoduction">Timeseries DataMap Introduction</a></li>
<li><a href="#compacting-pre-aggregate-tables">Compaction</a></li>
<li><a href="#data-management-with-pre-aggregate-tables">Data Management</a></li>
</ul>
<h2>
-<a id="timeseries-datamap-intoduction-alpha-feature-in-130" class="anchor" href="#timeseries-datamap-intoduction-alpha-feature-in-130" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Timeseries DataMap Intoduction (Alpha feature in 1.3.0)</h2>
-<p>Timeseries DataMap a pre-aggregate table implementation based on 'preaggregate' DataMap.
-Difference is that Timerseries DataMap has built-in understanding of time hierarchy and
+<a id="timeseries-datamap-introduction-alpha-feature-in-130" class="anchor" href="#timeseries-datamap-introduction-alpha-feature-in-130" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Timeseries DataMap Introduction (Alpha feature in 1.3.0)</h2>
+<p>Timeseries DataMap a pre-aggregate table implementation based on 'pre-aggregate' DataMap.
+Difference is that Timeseries DataMap has built-in understanding of time hierarchy and
levels: year, month, day, hour, minute, so that it supports automatic roll-up in time dimension
for query.</p>
<p>The data loading, querying, compaction command and its behavior is the same as preaggregate DataMap.
http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/src/main/webapp/data-management-on-carbondata.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/data-management-on-carbondata.html b/src/main/webapp/data-management-on-carbondata.html
index a9c7926..c81b02c 100644
--- a/src/main/webapp/data-management-on-carbondata.html
+++ b/src/main/webapp/data-management-on-carbondata.html
@@ -195,11 +195,11 @@
<a id="create-table" class="anchor" href="#create-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CREATE TABLE</h2>
<p>This command can be used to create a CarbonData table by specifying the list of fields along with the table properties. You can also specify the location where the table needs to be stored.</p>
<pre><code>CREATE TABLE [IF NOT EXISTS] [db_name.]table_name[(col_name data_type , ...)]
-STORED BY 'carbondata'
+STORED AS carbondata
[TBLPROPERTIES (property_name=property_value, ...)]
[LOCATION 'path']
</code></pre>
-<p><strong>NOTE:</strong> CarbonData also supports "STORED AS carbondata". Find example code at <a href="https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonSessionExample.scala" target=_blank>CarbonSessionExample</a> in the CarbonData repo.</p>
+<p><strong>NOTE:</strong> CarbonData also supports "STORED AS carbondata" and "USING carbondata". Find example code at <a href="https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonSessionExample.scala" target=_blank>CarbonSessionExample</a> in the CarbonData repo.</p>
<h3>
<a id="usage-guidelines" class="anchor" href="#usage-guidelines" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Usage Guidelines</h3>
<p>Following are the guidelines for TBLPROPERTIES, CarbonData's additional table options can be set via carbon.properties.</p>
http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/src/main/webapp/sdk-guide.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/sdk-guide.html b/src/main/webapp/sdk-guide.html
index 08044d1..7128c6c 100644
--- a/src/main/webapp/sdk-guide.html
+++ b/src/main/webapp/sdk-guide.html
@@ -340,6 +340,14 @@ These SDK writer output contains just a carbondata and carbonindex files. No met
</tbody>
</table>
<h2>
+<a id="run-sql-on-files-directly" class="anchor" href="#run-sql-on-files-directly" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Run SQL on files directly</h2>
+<p>Instead of creating table and query it, you can also query that file directly with SQL.</p>
+<h3>
+<a id="example" class="anchor" href="#example" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Example</h3>
+<pre><code>SELECT * FROM carbonfile.`$Path`
+</code></pre>
+<p>Find example code at <a href="https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/scala/org/apache/carbondata/examples/DirectSQLExample.scala" target=_blank>DirectSQLExample</a> in the CarbonData repo.</p>
+<h2>
<a id="api-list" class="anchor" href="#api-list" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>API List</h2>
<h3>
<a id="class-orgapachecarbondatasdkfilecarbonwriterbuilder" class="anchor" href="#class-orgapachecarbondatasdkfilecarbonwriterbuilder" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Class org.apache.carbondata.sdk.file.CarbonWriterBuilder</h3>
@@ -578,17 +586,22 @@ External client can make use of this reader to read CarbonData files without Car
String path = "./testWriteFiles";
CarbonReader reader = CarbonReader
.builder(path, "_temp")
- .projection(new String[]{"name", "age"})
+ .projection(new String[]{"stringField", "shortField", "intField", "longField",
+ "doubleField", "boolField", "dateField", "timeField", "decimalField"})
.build();
// 2. Read data
+ long day = 24L * 3600 * 1000;
int i = 0;
while (reader.hasNext()) {
- Object[] row = (Object[]) reader.readNextRow();
- System.out.println(row[0] + "\t" + row[1]);
- i++;
+ Object[] row = (Object[]) reader.readNextRow();
+ System.out.println(String.format("%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t",
+ i, row[0], row[1], row[2], row[3], row[4], row[5],
+ new Date((day * ((int) row[6]))), new Timestamp((long) row[7] / 1000), row[8]
+ ));
+ i++;
}
-
+
// 3. Close this reader
reader.close();
</code></pre>
@@ -597,12 +610,25 @@ External client can make use of this reader to read CarbonData files without Car
<a id="api-list-1" class="anchor" href="#api-list-1" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>API List</h2>
<h3>
<a id="class-orgapachecarbondatasdkfilecarbonreader" class="anchor" href="#class-orgapachecarbondatasdkfilecarbonreader" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Class org.apache.carbondata.sdk.file.CarbonReader</h3>
-<pre><code> /**
- * Return a new CarbonReaderBuilder instance
- */
+<pre><code> /**
+ * Return a new {@link CarbonReaderBuilder} instance
+ *
+ * @param tablePath table store path
+ * @param tableName table name
+ * @return CarbonReaderBuilder object
+ */
public static CarbonReaderBuilder builder(String tablePath, String tableName);
</code></pre>
<pre><code> /**
+ * Return a new CarbonReaderBuilder instance
+ * Default value of table name is table + tablePath + time
+ *
+ * @param tablePath table path
+ * @return CarbonReaderBuilder object
+ */
+ public static CarbonReaderBuilder builder(String tablePath);
+</code></pre>
+<pre><code> /**
* Return true if has next row
*/
public boolean hasNext();
http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/src/main/webapp/timeseries-datamap-guide.html
----------------------------------------------------------------------
diff --git a/src/main/webapp/timeseries-datamap-guide.html b/src/main/webapp/timeseries-datamap-guide.html
index b40150c..2400ce0 100644
--- a/src/main/webapp/timeseries-datamap-guide.html
+++ b/src/main/webapp/timeseries-datamap-guide.html
@@ -179,14 +179,14 @@
<div><h1>
<a id="carbondata-timeseries-datamap" class="anchor" href="#carbondata-timeseries-datamap" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CarbonData Timeseries DataMap</h1>
<ul>
-<li><a href="#timeseries-datamap-intoduction-(alpha-feature-in-1.3.0)">Timeseries DataMap</a></li>
+<li><a href="#timeseries-datamap-intoduction">Timeseries DataMap Introduction</a></li>
<li><a href="#compacting-pre-aggregate-tables">Compaction</a></li>
<li><a href="#data-management-with-pre-aggregate-tables">Data Management</a></li>
</ul>
<h2>
-<a id="timeseries-datamap-intoduction-alpha-feature-in-130" class="anchor" href="#timeseries-datamap-intoduction-alpha-feature-in-130" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Timeseries DataMap Intoduction (Alpha feature in 1.3.0)</h2>
-<p>Timeseries DataMap a pre-aggregate table implementation based on 'preaggregate' DataMap.
-Difference is that Timerseries DataMap has built-in understanding of time hierarchy and
+<a id="timeseries-datamap-introduction-alpha-feature-in-130" class="anchor" href="#timeseries-datamap-introduction-alpha-feature-in-130" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Timeseries DataMap Introduction (Alpha feature in 1.3.0)</h2>
+<p>Timeseries DataMap a pre-aggregate table implementation based on 'pre-aggregate' DataMap.
+Difference is that Timeseries DataMap has built-in understanding of time hierarchy and
levels: year, month, day, hour, minute, so that it supports automatic roll-up in time dimension
for query.</p>
<p>The data loading, querying, compaction command and its behavior is the same as preaggregate DataMap.
http://git-wip-us.apache.org/repos/asf/carbondata-site/blob/44fe8a42/src/site/markdown/sdk-guide.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/sdk-guide.md b/src/site/markdown/sdk-guide.md
index 1d225a9..2371b33 100644
--- a/src/site/markdown/sdk-guide.md
+++ b/src/site/markdown/sdk-guide.md
@@ -128,7 +128,14 @@ Each of SQL data types are mapped into data types of SDK. Following are the mapp
| STRING | DataTypes.STRING |
| DECIMAL | DataTypes.createDecimalType(precision, scale) |
+## Run SQL on files directly
+Instead of creating table and query it, you can also query that file directly with SQL.
+### Example
+```
+SELECT * FROM carbonfile.`$Path`
+```
+Find example code at [DirectSQLExample](https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/scala/org/apache/carbondata/examples/DirectSQLExample.scala) in the CarbonData repo.
## API List
### Class org.apache.carbondata.sdk.file.CarbonWriterBuilder
@@ -408,17 +415,22 @@ External client can make use of this reader to read CarbonData files without Car
String path = "./testWriteFiles";
CarbonReader reader = CarbonReader
.builder(path, "_temp")
- .projection(new String[]{"name", "age"})
+ .projection(new String[]{"stringField", "shortField", "intField", "longField",
+ "doubleField", "boolField", "dateField", "timeField", "decimalField"})
.build();
// 2. Read data
+ long day = 24L * 3600 * 1000;
int i = 0;
while (reader.hasNext()) {
- Object[] row = (Object[]) reader.readNextRow();
- System.out.println(row[0] + "\t" + row[1]);
- i++;
+ Object[] row = (Object[]) reader.readNextRow();
+ System.out.println(String.format("%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t",
+ i, row[0], row[1], row[2], row[3], row[4], row[5],
+ new Date((day * ((int) row[6]))), new Timestamp((long) row[7] / 1000), row[8]
+ ));
+ i++;
}
-
+
// 3. Close this reader
reader.close();
```
@@ -429,14 +441,29 @@ Find example code at [CarbonReaderExample](https://github.com/apache/carbondata/
### Class org.apache.carbondata.sdk.file.CarbonReader
```
- /**
- * Return a new CarbonReaderBuilder instance
- */
+ /**
+ * Return a new {@link CarbonReaderBuilder} instance
+ *
+ * @param tablePath table store path
+ * @param tableName table name
+ * @return CarbonReaderBuilder object
+ */
public static CarbonReaderBuilder builder(String tablePath, String tableName);
```
```
/**
+ * Return a new CarbonReaderBuilder instance
+ * Default value of table name is table + tablePath + time
+ *
+ * @param tablePath table path
+ * @return CarbonReaderBuilder object
+ */
+ public static CarbonReaderBuilder builder(String tablePath);
+```
+
+```
+ /**
* Return true if has next row
*/
public boolean hasNext();