You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@carbondata.apache.org by ch...@apache.org on 2021/07/28 06:21:11 UTC
[carbondata-site] branch asf-site updated: add new files to src
This is an automated email from the ASF dual-hosted git repository.
chenliang613 pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/carbondata-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 9cd314d add new files to src
9cd314d is described below
commit 9cd314df6aeab7f9e3bec5eac153e1a6d2d0c63a
Author: chenliang613 <ch...@apache.org>
AuthorDate: Wed Jul 28 14:20:55 2021 +0800
add new files to src
---
.../classes/CleanUp$$anonfun$cleanUp$1.class | Bin 0 -> 1048 bytes
src/main/webapp/WEB-INF/classes/CleanUp$.class | Bin 0 -> 3370 bytes
src/main/webapp/WEB-INF/classes/CleanUp.class | Bin 0 -> 719 bytes
.../classes/MDFileConverter$$anonfun$1.class | Bin 0 -> 1606 bytes
.../classes/MDFileConverter$$anonfun$2.class | Bin 0 -> 1617 bytes
.../webapp/WEB-INF/classes/MDFileConverter.class | Bin 0 -> 10608 bytes
.../webapp/WEB-INF/classes/META-INF/DEPENDENCIES | 113 +++++
src/main/webapp/WEB-INF/classes/META-INF/LICENSE | 202 +++++++++
src/main/webapp/WEB-INF/classes/META-INF/NOTICE | 8 +
...andler$$anonfun$convertReadMeExtensions$1.class | Bin 0 -> 2104 bytes
.../webapp/WEB-INF/classes/MdFileHandler.class | Bin 0 -> 6285 bytes
src/main/webapp/WEB-INF/classes/Module.class | Bin 0 -> 1112 bytes
...siteLauncher$$anonfun$deleteRecursively$1.class | Bin 0 -> 1133 bytes
.../webapp/WEB-INF/classes/WebsiteLauncher$.class | Bin 0 -> 4248 bytes
.../webapp/WEB-INF/classes/WebsiteLauncher.class | Bin 0 -> 846 bytes
src/main/webapp/WEB-INF/classes/application.conf | 68 +++
src/main/webapp/WEB-INF/classes/html/footer.html | 16 +
src/main/webapp/WEB-INF/classes/html/header.html | 215 +++++++++
.../WEB-INF/classes/services/ConfService.class | Bin 0 -> 825 bytes
.../WEB-INF/classes/services/ConfServiceImpl.class | Bin 0 -> 2905 bytes
.../WEB-INF/classes/services/DataService.class | Bin 0 -> 781 bytes
.../WEB-INF/classes/services/DataServiceImpl.class | Bin 0 -> 4358 bytes
.../WEB-INF/classes/services/FileService.class | Bin 0 -> 708 bytes
.../WEB-INF/classes/services/FileServiceImpl.class | Bin 0 -> 3052 bytes
src/main/webapp/WEB-INF/lib/aopalliance-1.0.jar | Bin 0 -> 4467 bytes
.../webapp/WEB-INF/lib/classworlds-1.1-alpha-2.jar | Bin 0 -> 37516 bytes
src/main/webapp/WEB-INF/lib/commons-codec-1.9.jar | Bin 0 -> 263965 bytes
.../webapp/WEB-INF/lib/commons-logging-1.2.jar | Bin 0 -> 61829 bytes
src/main/webapp/WEB-INF/lib/config-1.3.1.jar | Bin 0 -> 282549 bytes
src/main/webapp/WEB-INF/lib/guice-3.0.jar | Bin 0 -> 710492 bytes
src/main/webapp/WEB-INF/lib/httpclient-4.5.3.jar | Bin 0 -> 747794 bytes
src/main/webapp/WEB-INF/lib/httpcore-4.4.6.jar | Bin 0 -> 323824 bytes
src/main/webapp/WEB-INF/lib/javax.inject-1.jar | Bin 0 -> 2497 bytes
src/main/webapp/WEB-INF/lib/junit-3.8.1.jar | Bin 0 -> 121070 bytes
src/main/webapp/WEB-INF/lib/maven-artifact-3.0.jar | Bin 0 -> 51908 bytes
src/main/webapp/WEB-INF/lib/maven-model-3.0.jar | Bin 0 -> 164685 bytes
.../webapp/WEB-INF/lib/maven-plugin-api-3.0.jar | Bin 0 -> 48920 bytes
src/main/webapp/WEB-INF/lib/mockito-all-1.9.5.jar | Bin 0 -> 1581066 bytes
.../WEB-INF/lib/plexus-classworlds-2.2.3.jar | Bin 0 -> 46099 bytes
.../lib/plexus-component-annotations-1.5.4.jar | Bin 0 -> 4208 bytes
...exus-container-default-1.0-alpha-9-stable-1.jar | Bin 0 -> 194185 bytes
.../WEB-INF/lib/plexus-resources-1.0-alpha-7.jar | Bin 0 -> 23259 bytes
src/main/webapp/WEB-INF/lib/plexus-utils-1.5.5.jar | Bin 0 -> 251275 bytes
.../webapp/WEB-INF/lib/scala-library-2.12.1.jar | Bin 0 -> 5272325 bytes
.../lib/scala-parser-combinators_2.11-1.0.1.jar | Bin 0 -> 419701 bytes
.../lib/scala-parser-combinators_2.12-1.0.4.jar | Bin 0 -> 204313 bytes
.../webapp/WEB-INF/lib/scala-reflect-2.12.0.jar | Bin 0 -> 3574999 bytes
.../webapp/WEB-INF/lib/scala-xml_2.11-1.0.1.jar | Bin 0 -> 655685 bytes
.../webapp/WEB-INF/lib/scala-xml_2.12-1.0.5.jar | Bin 0 -> 548430 bytes
.../webapp/WEB-INF/lib/scalactic_2.12-3.0.1.jar | Bin 0 -> 705564 bytes
.../webapp/WEB-INF/lib/scalariform_2.11-0.1.7.jar | Bin 0 -> 1934192 bytes
.../WEB-INF/lib/scalastyle-maven-plugin-0.8.0.jar | Bin 0 -> 18780 bytes
.../webapp/WEB-INF/lib/scalastyle_2.11-0.8.0.jar | Bin 0 -> 894844 bytes
.../webapp/WEB-INF/lib/scalatest_2.12-3.0.1.jar | Bin 0 -> 6992597 bytes
.../webapp/WEB-INF/lib/sisu-guice-2.1.7-noaop.jar | Bin 0 -> 471737 bytes
.../webapp/WEB-INF/lib/sisu-inject-bean-1.4.2.jar | Bin 0 -> 152989 bytes
.../WEB-INF/lib/sisu-inject-plexus-1.4.2.jar | Bin 0 -> 201716 bytes
src/main/webapp/WEB-INF/lib/slf4j-api-1.7.25.jar | Bin 0 -> 41203 bytes
.../webapp/WEB-INF/lib/slf4j-simple-1.7.25.jar | Bin 0 -> 15257 bytes
src/main/webapp/bloomfilter-datamap-guide.html | 392 ++++++++++++++++
src/main/webapp/css/style.css | 4 +-
src/main/webapp/datamap-developer-guide.html | 266 +++++++++++
src/main/webapp/datamap-management.html | 394 ++++++++++++++++
src/main/webapp/index.html | 37 +-
src/main/webapp/lucene-datamap-guide.html | 389 ++++++++++++++++
.../webapp/pdf/TPCDS_Report_CarbonData2.1.0.pdf | Bin 0 -> 717503 bytes
src/main/webapp/preaggregate-datamap-guide.html | 502 +++++++++++++++++++++
src/main/webapp/timeseries-datamap-guide.html | 369 +++++++++++++++
68 files changed, 2954 insertions(+), 21 deletions(-)
diff --git a/src/main/webapp/WEB-INF/classes/CleanUp$$anonfun$cleanUp$1.class b/src/main/webapp/WEB-INF/classes/CleanUp$$anonfun$cleanUp$1.class
new file mode 100644
index 0000000..5b6d7e1
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/CleanUp$$anonfun$cleanUp$1.class differ
diff --git a/src/main/webapp/WEB-INF/classes/CleanUp$.class b/src/main/webapp/WEB-INF/classes/CleanUp$.class
new file mode 100644
index 0000000..099f826
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/CleanUp$.class differ
diff --git a/src/main/webapp/WEB-INF/classes/CleanUp.class b/src/main/webapp/WEB-INF/classes/CleanUp.class
new file mode 100644
index 0000000..5b3bfaa
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/CleanUp.class differ
diff --git a/src/main/webapp/WEB-INF/classes/MDFileConverter$$anonfun$1.class b/src/main/webapp/WEB-INF/classes/MDFileConverter$$anonfun$1.class
new file mode 100644
index 0000000..a1c473f
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/MDFileConverter$$anonfun$1.class differ
diff --git a/src/main/webapp/WEB-INF/classes/MDFileConverter$$anonfun$2.class b/src/main/webapp/WEB-INF/classes/MDFileConverter$$anonfun$2.class
new file mode 100644
index 0000000..157c615
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/MDFileConverter$$anonfun$2.class differ
diff --git a/src/main/webapp/WEB-INF/classes/MDFileConverter.class b/src/main/webapp/WEB-INF/classes/MDFileConverter.class
new file mode 100644
index 0000000..7cce73f
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/MDFileConverter.class differ
diff --git a/src/main/webapp/WEB-INF/classes/META-INF/DEPENDENCIES b/src/main/webapp/WEB-INF/classes/META-INF/DEPENDENCIES
new file mode 100644
index 0000000..dd8223b
--- /dev/null
+++ b/src/main/webapp/WEB-INF/classes/META-INF/DEPENDENCIES
@@ -0,0 +1,113 @@
+// ------------------------------------------------------------------
+// Transitive dependencies of this project determined from the
+// maven pom organized by organization.
+// ------------------------------------------------------------------
+
+Apache CarbonData :: Website
+
+
+From: 'an unknown organization'
+ - AOP alliance (http://aopalliance.sourceforge.net) aopalliance:aopalliance:jar:1.0
+ License: Public Domain
+ - javax.inject (http://code.google.com/p/atinject/) javax.inject:javax.inject:jar:1
+ License: The Apache Software License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+ - Mockito (http://www.mockito.org) org.mockito:mockito-all:jar:1.9.5
+ License: The MIT License (http://code.google.com/p/mockito/wiki/License)
+ - Sisu - Guice (http://forge.sonatype.com/sisu-guice/) org.sonatype.sisu:sisu-guice:pom:2.1.7
+ License: The Apache Software License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+
+From: 'Codehaus' (http://www.codehaus.org/)
+ - Plexus Classworlds (http://plexus.codehaus.org/plexus-classworlds/) org.codehaus.plexus:plexus-classworlds:jar:2.2.3
+ License: The Apache Software License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+ - Plexus :: Component Annotations (http://plexus.codehaus.org/plexus-containers/plexus-component-annotations/) org.codehaus.plexus:plexus-component-annotations:jar:1.5.4
+ License: The Apache Software License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+ - Default Plexus Container org.codehaus.plexus:plexus-container-default:jar:1.0-alpha-9-stable-1
+
+ - Plexus Resource Component (http://plexus.codehaus.org/plexus-components/plexus-resources) org.codehaus.plexus:plexus-resources:jar:1.0-alpha-7
+ License: The Apache Software License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+ - Plexus Common Utilities (http://plexus.codehaus.org/plexus-utils) org.codehaus.plexus:plexus-utils:jar:1.5.5
+ License: The Apache Software License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+
+From: 'com.typesafe'
+ - config (https://github.com/typesafehub/config) com.typesafe:config:bundle:1.3.1
+ License: Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0)
+
+From: 'Google, Inc.' (http://www.google.com)
+ - Google Guice - Core Library (http://code.google.com/p/google-guice/guice/) com.google.inject:guice:jar:3.0
+ License: The Apache Software License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+
+From: 'JUnit' (http://www.junit.org)
+ - JUnit (http://junit.org) junit:junit:jar:3.8.1
+ License: Common Public License Version 1.0 (http://www.opensource.org/licenses/cpl1.0.txt)
+
+From: 'LAMP/EPFL' (http://lamp.epfl.ch/)
+ - Scala Library (http://www.scala-lang.org/) org.scala-lang:scala-library:jar:2.12.1
+ License: BSD 3-Clause (http://www.scala-lang.org/license.html)
+ - Scala Compiler (http://www.scala-lang.org/) org.scala-lang:scala-reflect:jar:2.12.0
+ License: BSD 3-Clause (http://www.scala-lang.org/license.html)
+
+From: 'org.scala-lang.modules' (http://www.scala-lang.org/)
+ - scala-parser-combinators (http://www.scala-lang.org/) org.scala-lang.modules:scala-parser-combinators_2.11:bundle:1.0.1
+ License: BSD 3-clause (http://opensource.org/licenses/BSD-3-Clause)
+ - scala-parser-combinators (http://www.scala-lang.org/) org.scala-lang.modules:scala-parser-combinators_2.12:bundle:1.0.4
+ License: BSD 3-clause (http://opensource.org/licenses/BSD-3-Clause)
+ - scala-xml (http://www.scala-lang.org/) org.scala-lang.modules:scala-xml_2.11:bundle:1.0.1
+ License: BSD 3-clause (http://opensource.org/licenses/BSD-3-Clause)
+ - scala-xml (http://www.scala-lang.org/) org.scala-lang.modules:scala-xml_2.12:bundle:1.0.5
+ License: BSD 3-clause (http://opensource.org/licenses/BSD-3-Clause)
+
+From: 'org.scalactic'
+ - scalactic (http://www.scalatest.org) org.scalactic:scalactic_2.12:bundle:3.0.1
+ License: the Apache License, ASL Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0)
+
+From: 'org.scalariform'
+ - scalariform (http://github.com/mdr/scalariform) org.scalariform:scalariform_2.11:jar:0.1.7
+ License: MIT License (http://www.opensource.org/licenses/mit-license.php)
+
+From: 'org.scalastyle'
+ - scalastyle (http://www.scalastyle.org) org.scalastyle:scalastyle_2.11:jar:0.8.0
+ License: Apache-2.0 (http://www.apache.org/licenses/LICENSE-2.0.html)
+
+From: 'org.scalatest'
+ - scalatest (http://www.scalatest.org) org.scalatest:scalatest_2.12:bundle:3.0.1
+ License: the Apache License, ASL Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0)
+
+From: 'QOS.ch' (http://www.qos.ch)
+ - SLF4J API Module (http://www.slf4j.org) org.slf4j:slf4j-api:jar:1.7.25
+ License: MIT License (http://www.opensource.org/licenses/mit-license.php)
+ - SLF4J Simple Binding (http://www.slf4j.org) org.slf4j:slf4j-simple:jar:1.7.25
+ License: MIT License (http://www.opensource.org/licenses/mit-license.php)
+
+From: 'Scalastyle Organization' (https://github.com/scalastyle)
+ - Scalastyle Maven Plugin (http://www.scalastyle.org) org.scalastyle:scalastyle-maven-plugin:maven-plugin:0.8.0
+ License: Apache 2.0 (http://www.apache.org/licenses/LICENSE-2.0.html)
+
+From: 'Sonatype, Inc.' (http://www.sonatype.com)
+ - Sisu - Inject (JSR330 bean support) (http://sisu.sonatype.org/sisu-inject/guice-bean/sisu-inject-bean/) org.sonatype.sisu:sisu-inject-bean:bundle:1.4.2
+ License: The Apache Software License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+ - Sisu - Inject (Plexus bean support) (http://sisu.sonatype.org/sisu-inject/guice-bean/guice-plexus/sisu-inject-plexus/) org.sonatype.sisu:sisu-inject-plexus:bundle:1.4.2
+ License: The Apache Software License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+
+From: 'The Apache Software Foundation' (http://www.apache.org/)
+ - Apache Commons Codec (http://commons.apache.org/proper/commons-codec/) commons-codec:commons-codec:jar:1.9
+ License: The Apache Software License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+ - Apache Commons Logging (http://commons.apache.org/proper/commons-logging/) commons-logging:commons-logging:jar:1.2
+ License: The Apache Software License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+ - Apache HttpClient (http://hc.apache.org/httpcomponents-client) org.apache.httpcomponents:httpclient:jar:4.5.3
+ License: Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+ - Apache HttpCore (http://hc.apache.org/httpcomponents-core-ga) org.apache.httpcomponents:httpcore:jar:4.4.6
+ License: Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+ - Maven Artifact (http://maven.apache.org/maven-artifact/) org.apache.maven:maven-artifact:jar:3.0
+ License: The Apache Software License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+ - Maven Model (http://maven.apache.org/maven-model/) org.apache.maven:maven-model:jar:3.0
+ License: The Apache Software License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+ - Maven Plugin API (http://maven.apache.org/maven-plugin-api/) org.apache.maven:maven-plugin-api:jar:3.0
+ License: The Apache Software License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+
+From: 'The Codehaus' (http://codehaus.org/)
+ - classworlds (http://classworlds.codehaus.org/) classworlds:classworlds:jar:1.1-alpha-2
+
+
+
+
+
diff --git a/src/main/webapp/WEB-INF/classes/META-INF/LICENSE b/src/main/webapp/WEB-INF/classes/META-INF/LICENSE
new file mode 100644
index 0000000..d645695
--- /dev/null
+++ b/src/main/webapp/WEB-INF/classes/META-INF/LICENSE
@@ -0,0 +1,202 @@
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
diff --git a/src/main/webapp/WEB-INF/classes/META-INF/NOTICE b/src/main/webapp/WEB-INF/classes/META-INF/NOTICE
new file mode 100644
index 0000000..9cf6291
--- /dev/null
+++ b/src/main/webapp/WEB-INF/classes/META-INF/NOTICE
@@ -0,0 +1,8 @@
+
+Apache CarbonData :: Website
+Copyright 2021 The Apache Software Foundation
+
+This product includes software developed at
+The Apache Software Foundation (http://www.apache.org/).
+
+
diff --git a/src/main/webapp/WEB-INF/classes/MdFileHandler$$anonfun$convertReadMeExtensions$1.class b/src/main/webapp/WEB-INF/classes/MdFileHandler$$anonfun$convertReadMeExtensions$1.class
new file mode 100644
index 0000000..bdd1312
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/MdFileHandler$$anonfun$convertReadMeExtensions$1.class differ
diff --git a/src/main/webapp/WEB-INF/classes/MdFileHandler.class b/src/main/webapp/WEB-INF/classes/MdFileHandler.class
new file mode 100644
index 0000000..d0cd5a8
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/MdFileHandler.class differ
diff --git a/src/main/webapp/WEB-INF/classes/Module.class b/src/main/webapp/WEB-INF/classes/Module.class
new file mode 100644
index 0000000..a0bcb7b
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/Module.class differ
diff --git a/src/main/webapp/WEB-INF/classes/WebsiteLauncher$$anonfun$deleteRecursively$1.class b/src/main/webapp/WEB-INF/classes/WebsiteLauncher$$anonfun$deleteRecursively$1.class
new file mode 100644
index 0000000..b7f7385
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/WebsiteLauncher$$anonfun$deleteRecursively$1.class differ
diff --git a/src/main/webapp/WEB-INF/classes/WebsiteLauncher$.class b/src/main/webapp/WEB-INF/classes/WebsiteLauncher$.class
new file mode 100644
index 0000000..9e36ad5
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/WebsiteLauncher$.class differ
diff --git a/src/main/webapp/WEB-INF/classes/WebsiteLauncher.class b/src/main/webapp/WEB-INF/classes/WebsiteLauncher.class
new file mode 100644
index 0000000..82fab5d
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/WebsiteLauncher.class differ
diff --git a/src/main/webapp/WEB-INF/classes/application.conf b/src/main/webapp/WEB-INF/classes/application.conf
new file mode 100644
index 0000000..0cc31c3
--- /dev/null
+++ b/src/main/webapp/WEB-INF/classes/application.conf
@@ -0,0 +1,68 @@
+fileList=["configuration-parameters",
+ "quick-start-guide",
+ "faq",
+ "file-structure-of-carbondata",
+ "streaming-guide",
+ "supported-data-types-in-carbondata",
+ "performance-tuning",
+ "sdk-guide",
+ "index-developer-guide",
+ "ddl-of-carbondata",
+ "dml-of-carbondata",
+ "documentation",
+ "language-manual",
+ "s3-guide",
+ "segment-management-on-carbondata",
+ "release-guide",
+ "how-to-contribute-to-apache-carbondata",
+ "introduction",
+ "usecases",
+ "csdk-guide",
+ "carbon-as-spark-datasource-guide",
+ "alluxio-guide",
+ "hive-guide",
+ "index-server",
+
+ "flink-integration-guide",
+ "mv-guide",
+ "prestodb-guide",
+ "prestosql-guide",
+ "scd-and-cdc-guide",
+ "spatial-index-guide",
+ "clean-files",
+
+
+ ]
+dataMapFileList=[
+ "bloomfilter-index-guide",
+ "lucene-index-guide",
+ "secondary-index-guide",
+ "index-management"]
+
+dataMapFilesUrl="https://raw.githubusercontent.com/apache/carbondata/master/docs/index/"
+
+fileListToRetain=["quick-start-guide"
+]
+headerPath="src/main/scala/html/header.html"
+footerPath="src/main/scala/html/footer.html"
+outputFileLocation="src/main/webapp/"
+apiUrl="https://raw.githubusercontent.com/apache/carbondata/master/docs/"
+mdLink="https://api.github.com/markdown/raw"
+mdFileLocation="src/site/markdown/"
+imagesFilesList=["file-structure-of-carbondata",
+"2-1_1",
+"2-2_1",
+"2-3_1",
+"2-3_2",
+"2-3_3",
+"2-3_4",
+"2-4_1",
+"2-5_1",
+"2-5_2",
+"2-5_3",
+"2-6_1",
+"carbon_data_file_structure_new",
+"carbon_data_format_new",
+"carbondata-performance"
+]
+scriptsPath="src/main/scala/scripts/"
\ No newline at end of file
diff --git a/src/main/webapp/WEB-INF/classes/html/footer.html b/src/main/webapp/WEB-INF/classes/html/footer.html
new file mode 100644
index 0000000..47aa819
--- /dev/null
+++ b/src/main/webapp/WEB-INF/classes/html/footer.html
@@ -0,0 +1,16 @@
+</div>
+</div>
+</div>
+</div>
+<div class="doc-footer">
+ <a href="#top" class="scroll-top">Top</a>
+</div>
+</div>
+</section>
+</div>
+</div>
+</div>
+</section><!-- End systemblock part -->
+<script src="js/custom.js"></script>
+</body>
+</html>
\ No newline at end of file
diff --git a/src/main/webapp/WEB-INF/classes/html/header.html b/src/main/webapp/WEB-INF/classes/html/header.html
new file mode 100644
index 0000000..aa01b11
--- /dev/null
+++ b/src/main/webapp/WEB-INF/classes/html/header.html
@@ -0,0 +1,215 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href='images/favicon.ico' rel='shortcut icon' type='image/x-icon'>
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>CarbonData</title>
+ <style>
+
+ </style>
+ <!-- Bootstrap -->
+
+ <link rel="stylesheet" href="css/bootstrap.min.css">
+ <link href="css/style.css" rel="stylesheet">
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.scom/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ <script src="js/jquery.min.js"></script>
+ <script src="js/bootstrap.min.js"></script>
+ <script defer src="https://use.fontawesome.com/releases/v5.0.8/js/all.js"></script>
+
+
+</head>
+<body>
+<header>
+ <nav class="navbar navbar-default navbar-custom cd-navbar-wrapper">
+ <div class="container">
+ <div class="navbar-header">
+ <button aria-controls="navbar" aria-expanded="false" data-target="#navbar" data-toggle="collapse"
+ class="navbar-toggle collapsed" type="button">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a href="index.html" class="logo">
+ <img src="images/CarbonDataLogo.png" alt="CarbonData logo" title="CarbocnData logo"/>
+ </a>
+ </div>
+ <div class="navbar-collapse collapse cd_navcontnt" id="navbar">
+ <ul class="nav navbar-nav navbar-right navlist-custom">
+ <li><a href="index.html" class="hidden-xs"><i class="fa fa-home" aria-hidden="true"></i> </a>
+ </li>
+ <li><a href="index.html" class="hidden-lg hidden-md hidden-sm">Home</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle " data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false"> Download <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.0/"
+ target="_blank">Apache CarbonData 1.5.0</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
+ target="_blank">Apache CarbonData 1.4.1</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
+ target="_blank">Apache CarbonData 1.4.0</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
+ target="_blank">Apache CarbonData 1.3.1</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.0/"
+ target="_blank">Apache CarbonData 1.3.0</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
+ target="_blank">Release Archive</a></li>
+ </ul>
+ </li>
+ <li><a href="documentation.html" class="active">Documentation</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false">Community <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://github.com/apache/carbondata/blob/master/docs/how-to-contribute-to-apache-carbondata.md"
+ target="_blank">Contributing to CarbonData</a></li>
+ <li>
+ <a href="https://github.com/apache/carbondata/blob/master/docs/release-guide.md"
+ target="_blank">Release Guide</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/PMC+and+Committers+member+list"
+ target="_blank">Project PMC and Committers</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=66850609"
+ target="_blank">CarbonData Meetups</a></li>
+ <li><a href="security.html">Apache CarbonData Security</a></li>
+ <li><a href="https://issues.apache.org/jira/browse/CARBONDATA" target="_blank">Apache
+ Jira</a></li>
+ <li><a href="videogallery.html">CarbonData Videos </a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="apache_link hidden-xs dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="hidden-lg hidden-md hidden-sm dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+
+ <li>
+ <a href="#" id="search-icon"><i class="fa fa-search" aria-hidden="true"></i></a>
+
+ </li>
+
+ </ul>
+ </div><!--/.nav-collapse -->
+ <div id="search-box">
+ <form method="get" action="http://www.google.com/search" target="_blank">
+ <div class="search-block">
+ <table border="0" cellpadding="0" width="100%">
+ <tr>
+ <td style="width:80%">
+ <input type="text" name="q" size=" 5" maxlength="255" value=""
+ class="search-input" placeholder="Search...." required/>
+ </td>
+ <td style="width:20%">
+ <input type="submit" value="Search"/></td>
+ </tr>
+ <tr>
+ <td align="left" style="font-size:75%" colspan="2">
+ <input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
+ </td>
+ </tr>
+ </table>
+ </div>
+ </form>
+ </div>
+ </div>
+ </nav>
+</header> <!-- end Header part -->
+
+<div class="fixed-padding"></div> <!-- top padding with fixde header -->
+
+<section><!-- Dashboard nav -->
+ <div class="container-fluid q">
+ <div class="col-sm-12 col-md-12 maindashboard">
+ <div class="verticalnavbar">
+ <nav class="b-sticky-nav">
+ <div class="nav-scroller">
+ <div class="nav__inner">
+ <a class="b-nav__intro nav__item" href="./introduction.html">introduction</a>
+ <a class="b-nav__quickstart nav__item" href="./quick-start-guide.html">quick start</a>
+ <a class="b-nav__uses nav__item" href="./usecases.html">use cases</a>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__docs nav__item nav__sub__anchor" href="./language-manual.html">Language Reference</a>
+ <a class="nav__item nav__sub__item" href="./ddl-of-carbondata.html">DDL</a>
+ <a class="nav__item nav__sub__item" href="./dml-of-carbondata.html">DML</a>
+ <a class="nav__item nav__sub__item" href="./streaming-guide.html">Streaming</a>
+ <a class="nav__item nav__sub__item" href="./configuration-parameters.html">Configuration</a>
+ <a class="nav__item nav__sub__item" href="./datamap-developer-guide.html">Datamaps</a>
+ <a class="nav__item nav__sub__item" href="./supported-data-types-in-carbondata.html">Data Types</a>
+ </div>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__datamap nav__item nav__sub__anchor" href="./datamap-management.html">DataMaps</a>
+ <a class="nav__item nav__sub__item" href="./bloomfilter-datamap-guide.html">Bloom Filter</a>
+ <a class="nav__item nav__sub__item" href="./lucene-datamap-guide.html">Lucene</a>
+ <a class="nav__item nav__sub__item" href="./preaggregate-datamap-guide.html">Pre-Aggregate</a>
+ <a class="nav__item nav__sub__item" href="./timeseries-datamap-guide.html">Time Series</a>
+ </div>
+
+ <a class="b-nav__api nav__item" href="./sdk-guide.html">API</a>
+ <a class="b-nav__perf nav__item" href="./performance-tuning.html">Performance Tuning</a>
+ <a class="b-nav__s3 nav__item" href="./s3-guide.html">S3 Storage</a>
+ <a class="b-nav__faq nav__item" href="./faq.html">FAQ</a>
+ <a class="b-nav__contri nav__item" href="./how-to-contribute-to-apache-carbondata.html">Contribute</a>
+ <a class="b-nav__security nav__item" href="./security.html">Security</a>
+ <a class="b-nav__release nav__item" href="./release-guide.html">Release Guide</a>
+ </div>
+ </div>
+ <div class="navindicator">
+ <div class="b-nav__intro navindicator__item"></div>
+ <div class="b-nav__quickstart navindicator__item"></div>
+ <div class="b-nav__uses navindicator__item"></div>
+ <div class="b-nav__docs navindicator__item"></div>
+ <div class="b-nav__datamap navindicator__item"></div>
+ <div class="b-nav__api navindicator__item"></div>
+ <div class="b-nav__perf navindicator__item"></div>
+ <div class="b-nav__s3 navindicator__item"></div>
+ <div class="b-nav__faq navindicator__item"></div>
+ <div class="b-nav__contri navindicator__item"></div>
+ <div class="b-nav__security navindicator__item"></div>
+ </div>
+ </nav>
+ </div>
+ <div class="mdcontent">
+ <section>
+ <div style="padding:10px 15px;">
+ <div id="viewpage" name="viewpage">
+ <div class="row">
+ <div class="col-sm-12 col-md-12">
+ <div>
\ No newline at end of file
diff --git a/src/main/webapp/WEB-INF/classes/services/ConfService.class b/src/main/webapp/WEB-INF/classes/services/ConfService.class
new file mode 100644
index 0000000..f38dfca
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/services/ConfService.class differ
diff --git a/src/main/webapp/WEB-INF/classes/services/ConfServiceImpl.class b/src/main/webapp/WEB-INF/classes/services/ConfServiceImpl.class
new file mode 100644
index 0000000..ca1af57
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/services/ConfServiceImpl.class differ
diff --git a/src/main/webapp/WEB-INF/classes/services/DataService.class b/src/main/webapp/WEB-INF/classes/services/DataService.class
new file mode 100644
index 0000000..4783e0d
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/services/DataService.class differ
diff --git a/src/main/webapp/WEB-INF/classes/services/DataServiceImpl.class b/src/main/webapp/WEB-INF/classes/services/DataServiceImpl.class
new file mode 100644
index 0000000..f99cef9
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/services/DataServiceImpl.class differ
diff --git a/src/main/webapp/WEB-INF/classes/services/FileService.class b/src/main/webapp/WEB-INF/classes/services/FileService.class
new file mode 100644
index 0000000..ddd46db
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/services/FileService.class differ
diff --git a/src/main/webapp/WEB-INF/classes/services/FileServiceImpl.class b/src/main/webapp/WEB-INF/classes/services/FileServiceImpl.class
new file mode 100644
index 0000000..0dadcf9
Binary files /dev/null and b/src/main/webapp/WEB-INF/classes/services/FileServiceImpl.class differ
diff --git a/src/main/webapp/WEB-INF/lib/aopalliance-1.0.jar b/src/main/webapp/WEB-INF/lib/aopalliance-1.0.jar
new file mode 100644
index 0000000..578b1a0
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/aopalliance-1.0.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/classworlds-1.1-alpha-2.jar b/src/main/webapp/WEB-INF/lib/classworlds-1.1-alpha-2.jar
new file mode 100644
index 0000000..1208b48
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/classworlds-1.1-alpha-2.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/commons-codec-1.9.jar b/src/main/webapp/WEB-INF/lib/commons-codec-1.9.jar
new file mode 100644
index 0000000..ef35f1c
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/commons-codec-1.9.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/commons-logging-1.2.jar b/src/main/webapp/WEB-INF/lib/commons-logging-1.2.jar
new file mode 100644
index 0000000..93a3b9f
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/commons-logging-1.2.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/config-1.3.1.jar b/src/main/webapp/WEB-INF/lib/config-1.3.1.jar
new file mode 100644
index 0000000..a21f42b
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/config-1.3.1.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/guice-3.0.jar b/src/main/webapp/WEB-INF/lib/guice-3.0.jar
new file mode 100644
index 0000000..f313e2b
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/guice-3.0.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/httpclient-4.5.3.jar b/src/main/webapp/WEB-INF/lib/httpclient-4.5.3.jar
new file mode 100644
index 0000000..8af4561
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/httpclient-4.5.3.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/httpcore-4.4.6.jar b/src/main/webapp/WEB-INF/lib/httpcore-4.4.6.jar
new file mode 100644
index 0000000..16ed0d1
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/httpcore-4.4.6.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/javax.inject-1.jar b/src/main/webapp/WEB-INF/lib/javax.inject-1.jar
new file mode 100644
index 0000000..b2a9d0b
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/javax.inject-1.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/junit-3.8.1.jar b/src/main/webapp/WEB-INF/lib/junit-3.8.1.jar
new file mode 100644
index 0000000..674d71e
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/junit-3.8.1.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/maven-artifact-3.0.jar b/src/main/webapp/WEB-INF/lib/maven-artifact-3.0.jar
new file mode 100644
index 0000000..824aec5
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/maven-artifact-3.0.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/maven-model-3.0.jar b/src/main/webapp/WEB-INF/lib/maven-model-3.0.jar
new file mode 100644
index 0000000..7993699
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/maven-model-3.0.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/maven-plugin-api-3.0.jar b/src/main/webapp/WEB-INF/lib/maven-plugin-api-3.0.jar
new file mode 100644
index 0000000..bbb4f00
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/maven-plugin-api-3.0.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/mockito-all-1.9.5.jar b/src/main/webapp/WEB-INF/lib/mockito-all-1.9.5.jar
new file mode 100644
index 0000000..00416eb
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/mockito-all-1.9.5.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/plexus-classworlds-2.2.3.jar b/src/main/webapp/WEB-INF/lib/plexus-classworlds-2.2.3.jar
new file mode 100644
index 0000000..182399c
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/plexus-classworlds-2.2.3.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/plexus-component-annotations-1.5.4.jar b/src/main/webapp/WEB-INF/lib/plexus-component-annotations-1.5.4.jar
new file mode 100644
index 0000000..bfb4895
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/plexus-component-annotations-1.5.4.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/plexus-container-default-1.0-alpha-9-stable-1.jar b/src/main/webapp/WEB-INF/lib/plexus-container-default-1.0-alpha-9-stable-1.jar
new file mode 100644
index 0000000..d205236
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/plexus-container-default-1.0-alpha-9-stable-1.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/plexus-resources-1.0-alpha-7.jar b/src/main/webapp/WEB-INF/lib/plexus-resources-1.0-alpha-7.jar
new file mode 100644
index 0000000..0e0b782
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/plexus-resources-1.0-alpha-7.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/plexus-utils-1.5.5.jar b/src/main/webapp/WEB-INF/lib/plexus-utils-1.5.5.jar
new file mode 100644
index 0000000..ec865ba
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/plexus-utils-1.5.5.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/scala-library-2.12.1.jar b/src/main/webapp/WEB-INF/lib/scala-library-2.12.1.jar
new file mode 100644
index 0000000..5396741
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/scala-library-2.12.1.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/scala-parser-combinators_2.11-1.0.1.jar b/src/main/webapp/WEB-INF/lib/scala-parser-combinators_2.11-1.0.1.jar
new file mode 100644
index 0000000..fd758c8
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/scala-parser-combinators_2.11-1.0.1.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/scala-parser-combinators_2.12-1.0.4.jar b/src/main/webapp/WEB-INF/lib/scala-parser-combinators_2.12-1.0.4.jar
new file mode 100644
index 0000000..56fa393
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/scala-parser-combinators_2.12-1.0.4.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/scala-reflect-2.12.0.jar b/src/main/webapp/WEB-INF/lib/scala-reflect-2.12.0.jar
new file mode 100644
index 0000000..bf7a29a
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/scala-reflect-2.12.0.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/scala-xml_2.11-1.0.1.jar b/src/main/webapp/WEB-INF/lib/scala-xml_2.11-1.0.1.jar
new file mode 100644
index 0000000..7a595de
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/scala-xml_2.11-1.0.1.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/scala-xml_2.12-1.0.5.jar b/src/main/webapp/WEB-INF/lib/scala-xml_2.12-1.0.5.jar
new file mode 100644
index 0000000..43cba8a
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/scala-xml_2.12-1.0.5.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/scalactic_2.12-3.0.1.jar b/src/main/webapp/WEB-INF/lib/scalactic_2.12-3.0.1.jar
new file mode 100644
index 0000000..0526f24
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/scalactic_2.12-3.0.1.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/scalariform_2.11-0.1.7.jar b/src/main/webapp/WEB-INF/lib/scalariform_2.11-0.1.7.jar
new file mode 100644
index 0000000..04cad4d
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/scalariform_2.11-0.1.7.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/scalastyle-maven-plugin-0.8.0.jar b/src/main/webapp/WEB-INF/lib/scalastyle-maven-plugin-0.8.0.jar
new file mode 100644
index 0000000..cc9df99
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/scalastyle-maven-plugin-0.8.0.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/scalastyle_2.11-0.8.0.jar b/src/main/webapp/WEB-INF/lib/scalastyle_2.11-0.8.0.jar
new file mode 100644
index 0000000..0acd99f
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/scalastyle_2.11-0.8.0.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/scalatest_2.12-3.0.1.jar b/src/main/webapp/WEB-INF/lib/scalatest_2.12-3.0.1.jar
new file mode 100644
index 0000000..1fa54ea
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/scalatest_2.12-3.0.1.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/sisu-guice-2.1.7-noaop.jar b/src/main/webapp/WEB-INF/lib/sisu-guice-2.1.7-noaop.jar
new file mode 100644
index 0000000..aa66844
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/sisu-guice-2.1.7-noaop.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/sisu-inject-bean-1.4.2.jar b/src/main/webapp/WEB-INF/lib/sisu-inject-bean-1.4.2.jar
new file mode 100644
index 0000000..ca6abdd
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/sisu-inject-bean-1.4.2.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/sisu-inject-plexus-1.4.2.jar b/src/main/webapp/WEB-INF/lib/sisu-inject-plexus-1.4.2.jar
new file mode 100644
index 0000000..53063c7
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/sisu-inject-plexus-1.4.2.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/slf4j-api-1.7.25.jar b/src/main/webapp/WEB-INF/lib/slf4j-api-1.7.25.jar
new file mode 100644
index 0000000..0143c09
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/slf4j-api-1.7.25.jar differ
diff --git a/src/main/webapp/WEB-INF/lib/slf4j-simple-1.7.25.jar b/src/main/webapp/WEB-INF/lib/slf4j-simple-1.7.25.jar
new file mode 100644
index 0000000..a7260f3
Binary files /dev/null and b/src/main/webapp/WEB-INF/lib/slf4j-simple-1.7.25.jar differ
diff --git a/src/main/webapp/bloomfilter-datamap-guide.html b/src/main/webapp/bloomfilter-datamap-guide.html
new file mode 100644
index 0000000..19ee42a
--- /dev/null
+++ b/src/main/webapp/bloomfilter-datamap-guide.html
@@ -0,0 +1,392 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href='images/favicon.ico' rel='shortcut icon' type='image/x-icon'>
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>CarbonData</title>
+ <style>
+
+ </style>
+ <!-- Bootstrap -->
+
+ <link rel="stylesheet" href="css/bootstrap.min.css">
+ <link href="css/style.css" rel="stylesheet">
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.scom/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ <script src="js/jquery.min.js"></script>
+ <script src="js/bootstrap.min.js"></script>
+ <script defer src="https://use.fontawesome.com/releases/v5.0.8/js/all.js"></script>
+
+
+</head>
+<body>
+<header>
+ <nav class="navbar navbar-default navbar-custom cd-navbar-wrapper">
+ <div class="container">
+ <div class="navbar-header">
+ <button aria-controls="navbar" aria-expanded="false" data-target="#navbar" data-toggle="collapse"
+ class="navbar-toggle collapsed" type="button">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a href="index.html" class="logo">
+ <img src="images/CarbonDataLogo.png" alt="CarbonData logo" title="CarbocnData logo"/>
+ </a>
+ </div>
+ <div class="navbar-collapse collapse cd_navcontnt" id="navbar">
+ <ul class="nav navbar-nav navbar-right navlist-custom">
+ <li><a href="index.html" class="hidden-xs"><i class="fa fa-home" aria-hidden="true"></i> </a>
+ </li>
+ <li><a href="index.html" class="hidden-lg hidden-md hidden-sm">Home</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle " data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false"> Download <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.0/"
+ target="_blank">Apache CarbonData 1.5.0</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
+ target="_blank">Apache CarbonData 1.4.1</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
+ target="_blank">Apache CarbonData 1.4.0</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
+ target="_blank">Apache CarbonData 1.3.1</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.0/"
+ target="_blank">Apache CarbonData 1.3.0</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
+ target="_blank">Release Archive</a></li>
+ </ul>
+ </li>
+ <li><a href="documentation.html" class="active">Documentation</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false">Community <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://github.com/apache/carbondata/blob/master/docs/how-to-contribute-to-apache-carbondata.md"
+ target="_blank">Contributing to CarbonData</a></li>
+ <li>
+ <a href="https://github.com/apache/carbondata/blob/master/docs/release-guide.md"
+ target="_blank">Release Guide</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/PMC+and+Committers+member+list"
+ target="_blank">Project PMC and Committers</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=66850609"
+ target="_blank">CarbonData Meetups</a></li>
+ <li><a href="security.html">Apache CarbonData Security</a></li>
+ <li><a href="https://issues.apache.org/jira/browse/CARBONDATA" target="_blank">Apache
+ Jira</a></li>
+ <li><a href="videogallery.html">CarbonData Videos </a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="apache_link hidden-xs dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="hidden-lg hidden-md hidden-sm dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+
+ <li>
+ <a href="#" id="search-icon"><i class="fa fa-search" aria-hidden="true"></i></a>
+
+ </li>
+
+ </ul>
+ </div><!--/.nav-collapse -->
+ <div id="search-box">
+ <form method="get" action="http://www.google.com/search" target="_blank">
+ <div class="search-block">
+ <table border="0" cellpadding="0" width="100%">
+ <tr>
+ <td style="width:80%">
+ <input type="text" name="q" size=" 5" maxlength="255" value=""
+ class="search-input" placeholder="Search...." required/>
+ </td>
+ <td style="width:20%">
+ <input type="submit" value="Search"/></td>
+ </tr>
+ <tr>
+ <td align="left" style="font-size:75%" colspan="2">
+ <input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
+ </td>
+ </tr>
+ </table>
+ </div>
+ </form>
+ </div>
+ </div>
+ </nav>
+</header> <!-- end Header part -->
+
+<div class="fixed-padding"></div> <!-- top padding with fixde header -->
+
+<section><!-- Dashboard nav -->
+ <div class="container-fluid q">
+ <div class="col-sm-12 col-md-12 maindashboard">
+ <div class="verticalnavbar">
+ <nav class="b-sticky-nav">
+ <div class="nav-scroller">
+ <div class="nav__inner">
+ <a class="b-nav__intro nav__item" href="./introduction.html">introduction</a>
+ <a class="b-nav__quickstart nav__item" href="./quick-start-guide.html">quick start</a>
+ <a class="b-nav__uses nav__item" href="./usecases.html">use cases</a>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__docs nav__item nav__sub__anchor" href="./language-manual.html">Language Reference</a>
+ <a class="nav__item nav__sub__item" href="./ddl-of-carbondata.html">DDL</a>
+ <a class="nav__item nav__sub__item" href="./dml-of-carbondata.html">DML</a>
+ <a class="nav__item nav__sub__item" href="./streaming-guide.html">Streaming</a>
+ <a class="nav__item nav__sub__item" href="./configuration-parameters.html">Configuration</a>
+ <a class="nav__item nav__sub__item" href="./datamap-developer-guide.html">Datamaps</a>
+ <a class="nav__item nav__sub__item" href="./supported-data-types-in-carbondata.html">Data Types</a>
+ </div>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__datamap nav__item nav__sub__anchor" href="./datamap-management.html">DataMaps</a>
+ <a class="nav__item nav__sub__item" href="./bloomfilter-datamap-guide.html">Bloom Filter</a>
+ <a class="nav__item nav__sub__item" href="./lucene-datamap-guide.html">Lucene</a>
+ <a class="nav__item nav__sub__item" href="./preaggregate-datamap-guide.html">Pre-Aggregate</a>
+ <a class="nav__item nav__sub__item" href="./timeseries-datamap-guide.html">Time Series</a>
+ </div>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__api nav__item nav__sub__anchor" href="./sdk-guide.html">API</a>
+ <a class="nav__item nav__sub__item" href="./sdk-guide.html">Java SDK</a>
+ <a class="nav__item nav__sub__item" href="./CSDK-guide.html">C++ SDK</a>
+ </div>
+
+ <a class="b-nav__perf nav__item" href="./performance-tuning.html">Performance Tuning</a>
+ <a class="b-nav__s3 nav__item" href="./s3-guide.html">S3 Storage</a>
+ <a class="b-nav__faq nav__item" href="./faq.html">FAQ</a>
+ <a class="b-nav__contri nav__item" href="./how-to-contribute-to-apache-carbondata.html">Contribute</a>
+ <a class="b-nav__security nav__item" href="./security.html">Security</a>
+ <a class="b-nav__release nav__item" href="./release-guide.html">Release Guide</a>
+ </div>
+ </div>
+ <div class="navindicator">
+ <div class="b-nav__intro navindicator__item"></div>
+ <div class="b-nav__quickstart navindicator__item"></div>
+ <div class="b-nav__uses navindicator__item"></div>
+ <div class="b-nav__docs navindicator__item"></div>
+ <div class="b-nav__datamap navindicator__item"></div>
+ <div class="b-nav__api navindicator__item"></div>
+ <div class="b-nav__perf navindicator__item"></div>
+ <div class="b-nav__s3 navindicator__item"></div>
+ <div class="b-nav__faq navindicator__item"></div>
+ <div class="b-nav__contri navindicator__item"></div>
+ <div class="b-nav__security navindicator__item"></div>
+ </div>
+ </nav>
+ </div>
+ <div class="mdcontent">
+ <section>
+ <div style="padding:10px 15px;">
+ <div id="viewpage" name="viewpage">
+ <div class="row">
+ <div class="col-sm-12 col-md-12">
+ <div>
+<h1>
+<a id="carbondata-bloomfilter-datamap" class="anchor" href="#carbondata-bloomfilter-datamap" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CarbonData BloomFilter DataMap</h1>
+<ul>
+<li><a href="#datamap-management">DataMap Management</a></li>
+<li><a href="#bloomfilter-datamap-introduction">BloomFilter Datamap Introduction</a></li>
+<li><a href="#loading-data">Loading Data</a></li>
+<li><a href="#querying-data">Querying Data</a></li>
+<li><a href="#data-management-with-bloomfilter-datamap">Data Management</a></li>
+<li><a href="#useful-tips">Useful Tips</a></li>
+</ul>
+<h4>
+<a id="datamap-management" class="anchor" href="#datamap-management" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>DataMap Management</h4>
+<p>Creating BloomFilter DataMap</p>
+<pre><code>CREATE DATAMAP [IF NOT EXISTS] datamap_name
+ON TABLE main_table
+USING 'bloomfilter'
+DMPROPERTIES ('index_columns'='city, name', 'BLOOM_SIZE'='640000', 'BLOOM_FPP'='0.00001')
+</code></pre>
+<p>Dropping specified datamap</p>
+<pre><code>DROP DATAMAP [IF EXISTS] datamap_name
+ON TABLE main_table
+</code></pre>
+<p>Showing all DataMaps on this table</p>
+<pre><code>SHOW DATAMAP
+ON TABLE main_table
+</code></pre>
+<p>Disable Datamap</p>
+<blockquote>
+<p>The datamap by default is enabled. To support tuning on query, we can disable a specific datamap during query to observe whether we can gain performance enhancement from it. This is effective only for current session.</p>
+</blockquote>
+<pre><code>// disable the datamap
+SET carbon.datamap.visible.dbName.tableName.dataMapName = false
+// enable the datamap
+SET carbon.datamap.visible.dbName.tableName.dataMapName = true
+</code></pre>
+<h2>
+<a id="bloomfilter-datamap-introduction" class="anchor" href="#bloomfilter-datamap-introduction" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>BloomFilter DataMap Introduction</h2>
+<p>A Bloom filter is a space-efficient probabilistic data structure that is used to test whether an element is a member of a set.
+Carbondata introduced BloomFilter as an index datamap to enhance the performance of querying with precise value.
+It is well suitable for queries that do precise match on high cardinality columns(such as Name/ID).
+Internally, CarbonData maintains a BloomFilter per blocklet for each index column to indicate that whether a value of the column is in this blocklet.
+Just like the other datamaps, BloomFilter datamap is managed along with main tables by CarbonData.
+User can create BloomFilter datamap on specified columns with specified BloomFilter configurations such as size and probability.</p>
+<p>For instance, main table called <strong>datamap_test</strong> which is defined as:</p>
+<pre><code>CREATE TABLE datamap_test (
+ id string,
+ name string,
+ age int,
+ city string,
+ country string)
+STORED AS carbondata
+TBLPROPERTIES('SORT_COLUMNS'='id')
+</code></pre>
+<p>In the above example, <code>id</code> and <code>name</code> are high cardinality columns
+and we always query on <code>id</code> and <code>name</code> with precise value.
+since <code>id</code> is in the sort_columns and it is orderd,
+query on it will be fast because CarbonData can skip all the irrelative blocklets.
+But queries on <code>name</code> may be bad since the blocklet minmax may not help,
+because in each blocklet the range of the value of <code>name</code> may be the same -- all from A* to z*.
+In this case, user can create a BloomFilter datamap on column <code>name</code>.
+Moreover, user can also create a BloomFilter datamap on the sort_columns.
+This is useful if user has too many segments and the range of the value of sort_columns are almost the same.</p>
+<p>User can create BloomFilter datamap using the Create DataMap DDL:</p>
+<pre><code>CREATE DATAMAP dm
+ON TABLE datamap_test
+USING 'bloomfilter'
+DMPROPERTIES ('INDEX_COLUMNS' = 'name,id', 'BLOOM_SIZE'='640000', 'BLOOM_FPP'='0.00001', 'BLOOM_COMPRESS'='true')
+</code></pre>
+<p><strong>Properties for BloomFilter DataMap</strong></p>
+<table>
+<thead>
+<tr>
+<th>Property</th>
+<th>Is Required</th>
+<th>Default Value</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>INDEX_COLUMNS</td>
+<td>YES</td>
+<td></td>
+<td>Carbondata will generate BloomFilter index on these columns. Queries on these columns are usually like 'COL = VAL'.</td>
+</tr>
+<tr>
+<td>BLOOM_SIZE</td>
+<td>NO</td>
+<td>640000</td>
+<td>This value is internally used by BloomFilter as the number of expected insertions, it will affect the size of BloomFilter index. Since each blocklet has a BloomFilter here, so the default value is the approximate distinct index values in a blocklet assuming that each blocklet contains 20 pages and each page contains 32000 records. The value should be an integer.</td>
+</tr>
+<tr>
+<td>BLOOM_FPP</td>
+<td>NO</td>
+<td>0.00001</td>
+<td>This value is internally used by BloomFilter as the False-Positive Probability, it will affect the size of bloomfilter index as well as the number of hash functions for the BloomFilter. The value should be in the range (0, 1). In one test scenario, a 96GB TPCH customer table with bloom_size=320000 and bloom_fpp=0.00001 will result in 18 false positive samples.</td>
+</tr>
+<tr>
+<td>BLOOM_COMPRESS</td>
+<td>NO</td>
+<td>true</td>
+<td>Whether to compress the BloomFilter index files.</td>
+</tr>
+</tbody>
+</table>
+<h2>
+<a id="loading-data" class="anchor" href="#loading-data" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Loading Data</h2>
+<p>When loading data to main table, BloomFilter files will be generated for all the
+index_columns given in DMProperties which contains the blockletId and a BloomFilter for each index column.
+These index files will be written inside a folder named with datamap name
+inside each segment folders.</p>
+<h2>
+<a id="querying-data" class="anchor" href="#querying-data" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Querying Data</h2>
+<p>User can verify whether a query can leverage BloomFilter datamap by executing <code>EXPLAIN</code> command,
+which will show the transformed logical plan, and thus user can check whether the BloomFilter datamap can skip blocklets during the scan.
+If the datamap does not prune blocklets well, you can try to increase the value of property <code>BLOOM_SIZE</code> and decrease the value of property <code>BLOOM_FPP</code>.</p>
+<h2>
+<a id="data-management-with-bloomfilter-datamap" class="anchor" href="#data-management-with-bloomfilter-datamap" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Data Management With BloomFilter DataMap</h2>
+<p>Data management with BloomFilter datamap has no difference with that on Lucene datamap.
+You can refer to the corresponding section in <code>CarbonData Lucene DataMap</code>.</p>
+<h2>
+<a id="useful-tips" class="anchor" href="#useful-tips" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Useful Tips</h2>
+<ul>
+<li>BloomFilter DataMap is suggested to be created on the high cardinality columns.
+Query conditions on these columns are always simple <code>equal</code> or <code>in</code>,
+such as 'col1=XX', 'col1 in (XX, YY)'.</li>
+<li>We can create multiple BloomFilter datamaps on one table,
+but we do recommend you to create one BloomFilter datamap that contains multiple index columns,
+because the data loading and query performance will be better.</li>
+<li>
+<code>BLOOM_FPP</code> is only the expected number from user, the actually FPP may be worse.
+If the BloomFilter datamap does not work well,
+you can try to increase <code>BLOOM_SIZE</code> and decrease <code>BLOOM_FPP</code> at the same time.
+Notice that bigger <code>BLOOM_SIZE</code> will increase the size of index file
+and smaller <code>BLOOM_FPP</code> will increase runtime calculation while performing query.</li>
+<li>'0' skipped blocklets of BloomFilter datamap in explain output indicates that
+BloomFilter datamap does not prune better than Main datamap.
+(For example since the data is not ordered, a specific value may be contained in many blocklets. In this case, bloom may not work better than Main DataMap.)
+If this occurs very often, it means that current BloomFilter is useless. You can disable or drop it.
+Sometimes we cannot see any pruning result about BloomFilter datamap in the explain output,
+this indicates that the previous datamap has pruned all the blocklets and there is no need to continue pruning.</li>
+<li>In some scenarios, the BloomFilter datamap may not enhance the query performance significantly
+but if it can reduce the number of spark task,
+there is still a chance that BloomFilter datamap can enhance the performance for concurrent query.</li>
+<li>Note that BloomFilter datamap will decrease the data loading performance and may cause slightly storage expansion (for datamap index file).</li>
+</ul>
+<script>
+$(function() {
+ // Show selected style on nav item
+ $('.b-nav__datamap').addClass('selected');
+
+ if (!$('.b-nav__datamap').parent().hasClass('nav__item__with__subs--expanded')) {
+ // Display datamap subnav items
+ $('.b-nav__datamap').parent().toggleClass('nav__item__with__subs--expanded');
+ }
+});
+</script></div>
+</div>
+</div>
+</div>
+<div class="doc-footer">
+ <a href="#top" class="scroll-top">Top</a>
+</div>
+</div>
+</section>
+</div>
+</div>
+</div>
+</section><!-- End systemblock part -->
+<script src="js/custom.js"></script>
+</body>
+</html>
\ No newline at end of file
diff --git a/src/main/webapp/css/style.css b/src/main/webapp/css/style.css
index 719f72d..ef82297 100644
--- a/src/main/webapp/css/style.css
+++ b/src/main/webapp/css/style.css
@@ -1308,8 +1308,8 @@ padding-left:30px;
}
@media screen and (min-width: 1690px) {
- .verticalnavbar{
- width: 11% !important;}
+ .verticalnavbar{
+ width: 11% !important;}
}
.verticalnavbar {
diff --git a/src/main/webapp/datamap-developer-guide.html b/src/main/webapp/datamap-developer-guide.html
new file mode 100644
index 0000000..f442fe2
--- /dev/null
+++ b/src/main/webapp/datamap-developer-guide.html
@@ -0,0 +1,266 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href='images/favicon.ico' rel='shortcut icon' type='image/x-icon'>
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>CarbonData</title>
+ <style>
+
+ </style>
+ <!-- Bootstrap -->
+
+ <link rel="stylesheet" href="css/bootstrap.min.css">
+ <link href="css/style.css" rel="stylesheet">
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.scom/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ <script src="js/jquery.min.js"></script>
+ <script src="js/bootstrap.min.js"></script>
+ <script defer src="https://use.fontawesome.com/releases/v5.0.8/js/all.js"></script>
+
+
+</head>
+<body>
+<header>
+ <nav class="navbar navbar-default navbar-custom cd-navbar-wrapper">
+ <div class="container">
+ <div class="navbar-header">
+ <button aria-controls="navbar" aria-expanded="false" data-target="#navbar" data-toggle="collapse"
+ class="navbar-toggle collapsed" type="button">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a href="index.html" class="logo">
+ <img src="images/CarbonDataLogo.png" alt="CarbonData logo" title="CarbocnData logo"/>
+ </a>
+ </div>
+ <div class="navbar-collapse collapse cd_navcontnt" id="navbar">
+ <ul class="nav navbar-nav navbar-right navlist-custom">
+ <li><a href="index.html" class="hidden-xs"><i class="fa fa-home" aria-hidden="true"></i> </a>
+ </li>
+ <li><a href="index.html" class="hidden-lg hidden-md hidden-sm">Home</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle " data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false"> Download <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.0/"
+ target="_blank">Apache CarbonData 1.5.0</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
+ target="_blank">Apache CarbonData 1.4.1</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
+ target="_blank">Apache CarbonData 1.4.0</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
+ target="_blank">Apache CarbonData 1.3.1</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.0/"
+ target="_blank">Apache CarbonData 1.3.0</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
+ target="_blank">Release Archive</a></li>
+ </ul>
+ </li>
+ <li><a href="documentation.html" class="active">Documentation</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false">Community <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://github.com/apache/carbondata/blob/master/docs/how-to-contribute-to-apache-carbondata.md"
+ target="_blank">Contributing to CarbonData</a></li>
+ <li>
+ <a href="https://github.com/apache/carbondata/blob/master/docs/release-guide.md"
+ target="_blank">Release Guide</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/PMC+and+Committers+member+list"
+ target="_blank">Project PMC and Committers</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=66850609"
+ target="_blank">CarbonData Meetups</a></li>
+ <li><a href="security.html">Apache CarbonData Security</a></li>
+ <li><a href="https://issues.apache.org/jira/browse/CARBONDATA" target="_blank">Apache
+ Jira</a></li>
+ <li><a href="videogallery.html">CarbonData Videos </a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="apache_link hidden-xs dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="hidden-lg hidden-md hidden-sm dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+
+ <li>
+ <a href="#" id="search-icon"><i class="fa fa-search" aria-hidden="true"></i></a>
+
+ </li>
+
+ </ul>
+ </div><!--/.nav-collapse -->
+ <div id="search-box">
+ <form method="get" action="http://www.google.com/search" target="_blank">
+ <div class="search-block">
+ <table border="0" cellpadding="0" width="100%">
+ <tr>
+ <td style="width:80%">
+ <input type="text" name="q" size=" 5" maxlength="255" value=""
+ class="search-input" placeholder="Search...." required/>
+ </td>
+ <td style="width:20%">
+ <input type="submit" value="Search"/></td>
+ </tr>
+ <tr>
+ <td align="left" style="font-size:75%" colspan="2">
+ <input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
+ </td>
+ </tr>
+ </table>
+ </div>
+ </form>
+ </div>
+ </div>
+ </nav>
+</header> <!-- end Header part -->
+
+<div class="fixed-padding"></div> <!-- top padding with fixde header -->
+
+<section><!-- Dashboard nav -->
+ <div class="container-fluid q">
+ <div class="col-sm-12 col-md-12 maindashboard">
+ <div class="verticalnavbar">
+ <nav class="b-sticky-nav">
+ <div class="nav-scroller">
+ <div class="nav__inner">
+ <a class="b-nav__intro nav__item" href="./introduction.html">introduction</a>
+ <a class="b-nav__quickstart nav__item" href="./quick-start-guide.html">quick start</a>
+ <a class="b-nav__uses nav__item" href="./usecases.html">use cases</a>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__docs nav__item nav__sub__anchor" href="./language-manual.html">Language Reference</a>
+ <a class="nav__item nav__sub__item" href="./ddl-of-carbondata.html">DDL</a>
+ <a class="nav__item nav__sub__item" href="./dml-of-carbondata.html">DML</a>
+ <a class="nav__item nav__sub__item" href="./streaming-guide.html">Streaming</a>
+ <a class="nav__item nav__sub__item" href="./configuration-parameters.html">Configuration</a>
+ <a class="nav__item nav__sub__item" href="./datamap-developer-guide.html">Datamaps</a>
+ <a class="nav__item nav__sub__item" href="./supported-data-types-in-carbondata.html">Data Types</a>
+ </div>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__datamap nav__item nav__sub__anchor" href="./datamap-management.html">DataMaps</a>
+ <a class="nav__item nav__sub__item" href="./bloomfilter-datamap-guide.html">Bloom Filter</a>
+ <a class="nav__item nav__sub__item" href="./lucene-datamap-guide.html">Lucene</a>
+ <a class="nav__item nav__sub__item" href="./preaggregate-datamap-guide.html">Pre-Aggregate</a>
+ <a class="nav__item nav__sub__item" href="./timeseries-datamap-guide.html">Time Series</a>
+ </div>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__api nav__item nav__sub__anchor" href="./sdk-guide.html">API</a>
+ <a class="nav__item nav__sub__item" href="./sdk-guide.html">Java SDK</a>
+ <a class="nav__item nav__sub__item" href="./CSDK-guide.html">C++ SDK</a>
+ </div>
+
+ <a class="b-nav__perf nav__item" href="./performance-tuning.html">Performance Tuning</a>
+ <a class="b-nav__s3 nav__item" href="./s3-guide.html">S3 Storage</a>
+ <a class="b-nav__faq nav__item" href="./faq.html">FAQ</a>
+ <a class="b-nav__contri nav__item" href="./how-to-contribute-to-apache-carbondata.html">Contribute</a>
+ <a class="b-nav__security nav__item" href="./security.html">Security</a>
+ <a class="b-nav__release nav__item" href="./release-guide.html">Release Guide</a>
+ </div>
+ </div>
+ <div class="navindicator">
+ <div class="b-nav__intro navindicator__item"></div>
+ <div class="b-nav__quickstart navindicator__item"></div>
+ <div class="b-nav__uses navindicator__item"></div>
+ <div class="b-nav__docs navindicator__item"></div>
+ <div class="b-nav__datamap navindicator__item"></div>
+ <div class="b-nav__api navindicator__item"></div>
+ <div class="b-nav__perf navindicator__item"></div>
+ <div class="b-nav__s3 navindicator__item"></div>
+ <div class="b-nav__faq navindicator__item"></div>
+ <div class="b-nav__contri navindicator__item"></div>
+ <div class="b-nav__security navindicator__item"></div>
+ </div>
+ </nav>
+ </div>
+ <div class="mdcontent">
+ <section>
+ <div style="padding:10px 15px;">
+ <div id="viewpage" name="viewpage">
+ <div class="row">
+ <div class="col-sm-12 col-md-12">
+ <div><h1>
+<a id="datamap-developer-guide" class="anchor" href="#datamap-developer-guide" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>DataMap Developer Guide</h1>
+<h3>
+<a id="introduction" class="anchor" href="#introduction" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Introduction</h3>
+<p>DataMap is a data structure that can be used to accelerate certain query of the table. Different DataMap can be implemented by developers.
+Currently, there are two 2 types of DataMap supported:</p>
+<ol>
+<li>IndexDataMap: DataMap that leverages index to accelerate filter query</li>
+<li>MVDataMap: DataMap that leverages Materialized View to accelerate olap style query, like SPJG query (select, predicate, join, groupby)</li>
+</ol>
+<h3>
+<a id="datamap-provider" class="anchor" href="#datamap-provider" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>DataMap Provider</h3>
+<p>When user issues <code>CREATE DATAMAP dm ON TABLE main USING 'provider'</code>, the corresponding DataMapProvider implementation will be created and initialized.
+Currently, the provider string can be:</p>
+<ol>
+<li>preaggregate: A type of MVDataMap that do pre-aggregate of single table</li>
+<li>timeseries: A type of MVDataMap that do pre-aggregate based on time dimension of the table</li>
+<li>class name IndexDataMapFactory implementation: Developer can implement new type of IndexDataMap by extending IndexDataMapFactory</li>
+</ol>
+<p>When user issues <code>DROP DATAMAP dm ON TABLE main</code>, the corresponding DataMapProvider interface will be called.</p>
+<p>Click for more details about <a href="./datamap-management.html#datamap-management">DataMap Management</a> and supported <a href="./datamap-management.html#overview">DSL</a>.</p>
+<script>
+$(function() {
+ // Show selected style on nav item
+ $('.b-nav__docs').addClass('selected');
+
+ // Display docs subnav items
+ if (!$('.b-nav__docs').parent().hasClass('nav__item__with__subs--expanded')) {
+ $('.b-nav__docs').parent().toggleClass('nav__item__with__subs--expanded');
+ }
+});
+</script></div>
+</div>
+</div>
+</div>
+<div class="doc-footer">
+ <a href="#top" class="scroll-top">Top</a>
+</div>
+</div>
+</section>
+</div>
+</div>
+</div>
+</section><!-- End systemblock part -->
+<script src="js/custom.js"></script>
+</body>
+</html>
\ No newline at end of file
diff --git a/src/main/webapp/datamap-management.html b/src/main/webapp/datamap-management.html
new file mode 100644
index 0000000..f5f9678
--- /dev/null
+++ b/src/main/webapp/datamap-management.html
@@ -0,0 +1,394 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href='images/favicon.ico' rel='shortcut icon' type='image/x-icon'>
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>CarbonData</title>
+ <style>
+
+ </style>
+ <!-- Bootstrap -->
+
+ <link rel="stylesheet" href="css/bootstrap.min.css">
+ <link href="css/style.css" rel="stylesheet">
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.scom/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ <script src="js/jquery.min.js"></script>
+ <script src="js/bootstrap.min.js"></script>
+ <script defer src="https://use.fontawesome.com/releases/v5.0.8/js/all.js"></script>
+
+
+</head>
+<body>
+<header>
+ <nav class="navbar navbar-default navbar-custom cd-navbar-wrapper">
+ <div class="container">
+ <div class="navbar-header">
+ <button aria-controls="navbar" aria-expanded="false" data-target="#navbar" data-toggle="collapse"
+ class="navbar-toggle collapsed" type="button">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a href="index.html" class="logo">
+ <img src="images/CarbonDataLogo.png" alt="CarbonData logo" title="CarbocnData logo"/>
+ </a>
+ </div>
+ <div class="navbar-collapse collapse cd_navcontnt" id="navbar">
+ <ul class="nav navbar-nav navbar-right navlist-custom">
+ <li><a href="index.html" class="hidden-xs"><i class="fa fa-home" aria-hidden="true"></i> </a>
+ </li>
+ <li><a href="index.html" class="hidden-lg hidden-md hidden-sm">Home</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle " data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false"> Download <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.0/"
+ target="_blank">Apache CarbonData 1.5.0</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
+ target="_blank">Apache CarbonData 1.4.1</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
+ target="_blank">Apache CarbonData 1.4.0</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
+ target="_blank">Apache CarbonData 1.3.1</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.0/"
+ target="_blank">Apache CarbonData 1.3.0</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
+ target="_blank">Release Archive</a></li>
+ </ul>
+ </li>
+ <li><a href="documentation.html" class="active">Documentation</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false">Community <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://github.com/apache/carbondata/blob/master/docs/how-to-contribute-to-apache-carbondata.md"
+ target="_blank">Contributing to CarbonData</a></li>
+ <li>
+ <a href="https://github.com/apache/carbondata/blob/master/docs/release-guide.md"
+ target="_blank">Release Guide</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/PMC+and+Committers+member+list"
+ target="_blank">Project PMC and Committers</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=66850609"
+ target="_blank">CarbonData Meetups</a></li>
+ <li><a href="security.html">Apache CarbonData Security</a></li>
+ <li><a href="https://issues.apache.org/jira/browse/CARBONDATA" target="_blank">Apache
+ Jira</a></li>
+ <li><a href="videogallery.html">CarbonData Videos </a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="apache_link hidden-xs dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="hidden-lg hidden-md hidden-sm dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+
+ <li>
+ <a href="#" id="search-icon"><i class="fa fa-search" aria-hidden="true"></i></a>
+
+ </li>
+
+ </ul>
+ </div><!--/.nav-collapse -->
+ <div id="search-box">
+ <form method="get" action="http://www.google.com/search" target="_blank">
+ <div class="search-block">
+ <table border="0" cellpadding="0" width="100%">
+ <tr>
+ <td style="width:80%">
+ <input type="text" name="q" size=" 5" maxlength="255" value=""
+ class="search-input" placeholder="Search...." required/>
+ </td>
+ <td style="width:20%">
+ <input type="submit" value="Search"/></td>
+ </tr>
+ <tr>
+ <td align="left" style="font-size:75%" colspan="2">
+ <input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
+ </td>
+ </tr>
+ </table>
+ </div>
+ </form>
+ </div>
+ </div>
+ </nav>
+</header> <!-- end Header part -->
+
+<div class="fixed-padding"></div> <!-- top padding with fixde header -->
+
+<section><!-- Dashboard nav -->
+ <div class="container-fluid q">
+ <div class="col-sm-12 col-md-12 maindashboard">
+ <div class="verticalnavbar">
+ <nav class="b-sticky-nav">
+ <div class="nav-scroller">
+ <div class="nav__inner">
+ <a class="b-nav__intro nav__item" href="./introduction.html">introduction</a>
+ <a class="b-nav__quickstart nav__item" href="./quick-start-guide.html">quick start</a>
+ <a class="b-nav__uses nav__item" href="./usecases.html">use cases</a>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__docs nav__item nav__sub__anchor" href="./language-manual.html">Language Reference</a>
+ <a class="nav__item nav__sub__item" href="./ddl-of-carbondata.html">DDL</a>
+ <a class="nav__item nav__sub__item" href="./dml-of-carbondata.html">DML</a>
+ <a class="nav__item nav__sub__item" href="./streaming-guide.html">Streaming</a>
+ <a class="nav__item nav__sub__item" href="./configuration-parameters.html">Configuration</a>
+ <a class="nav__item nav__sub__item" href="./datamap-developer-guide.html">Datamaps</a>
+ <a class="nav__item nav__sub__item" href="./supported-data-types-in-carbondata.html">Data Types</a>
+ </div>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__datamap nav__item nav__sub__anchor" href="./datamap-management.html">DataMaps</a>
+ <a class="nav__item nav__sub__item" href="./bloomfilter-datamap-guide.html">Bloom Filter</a>
+ <a class="nav__item nav__sub__item" href="./lucene-datamap-guide.html">Lucene</a>
+ <a class="nav__item nav__sub__item" href="./preaggregate-datamap-guide.html">Pre-Aggregate</a>
+ <a class="nav__item nav__sub__item" href="./timeseries-datamap-guide.html">Time Series</a>
+ </div>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__api nav__item nav__sub__anchor" href="./sdk-guide.html">API</a>
+ <a class="nav__item nav__sub__item" href="./sdk-guide.html">Java SDK</a>
+ <a class="nav__item nav__sub__item" href="./CSDK-guide.html">C++ SDK</a>
+ </div>
+
+ <a class="b-nav__perf nav__item" href="./performance-tuning.html">Performance Tuning</a>
+ <a class="b-nav__s3 nav__item" href="./s3-guide.html">S3 Storage</a>
+ <a class="b-nav__faq nav__item" href="./faq.html">FAQ</a>
+ <a class="b-nav__contri nav__item" href="./how-to-contribute-to-apache-carbondata.html">Contribute</a>
+ <a class="b-nav__security nav__item" href="./security.html">Security</a>
+ <a class="b-nav__release nav__item" href="./release-guide.html">Release Guide</a>
+ </div>
+ </div>
+ <div class="navindicator">
+ <div class="b-nav__intro navindicator__item"></div>
+ <div class="b-nav__quickstart navindicator__item"></div>
+ <div class="b-nav__uses navindicator__item"></div>
+ <div class="b-nav__docs navindicator__item"></div>
+ <div class="b-nav__datamap navindicator__item"></div>
+ <div class="b-nav__api navindicator__item"></div>
+ <div class="b-nav__perf navindicator__item"></div>
+ <div class="b-nav__s3 navindicator__item"></div>
+ <div class="b-nav__faq navindicator__item"></div>
+ <div class="b-nav__contri navindicator__item"></div>
+ <div class="b-nav__security navindicator__item"></div>
+ </div>
+ </nav>
+ </div>
+ <div class="mdcontent">
+ <section>
+ <div style="padding:10px 15px;">
+ <div id="viewpage" name="viewpage">
+ <div class="row">
+ <div class="col-sm-12 col-md-12">
+ <div>
+<h1>
+<a id="carbondata-datamap-management" class="anchor" href="#carbondata-datamap-management" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CarbonData DataMap Management</h1>
+<ul>
+<li><a href="#overview">Overview</a></li>
+<li><a href="#datamap-management">DataMap Management</a></li>
+<li><a href="#automatic-refresh">Automatic Refresh</a></li>
+<li><a href="#manual-refresh">Manual Refresh</a></li>
+<li><a href="#datamap-catalog">DataMap Catalog</a></li>
+<li>
+<a href="#datamap-related-commands">DataMap Related Commands</a>
+<ul>
+<li><a href="#explain">Explain</a></li>
+<li><a href="#show-datamap">Show DataMap</a></li>
+<li><a href="#compaction-on-datamap">Compaction on DataMap</a></li>
+</ul>
+</li>
+</ul>
+<h2>
+<a id="overview" class="anchor" href="#overview" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Overview</h2>
+<p>DataMap can be created using following DDL</p>
+<pre><code> CREATE DATAMAP [IF NOT EXISTS] datamap_name
+ [ON TABLE main_table]
+ USING "datamap_provider"
+ [WITH DEFERRED REBUILD]
+ DMPROPERTIES ('key'='value', ...)
+ AS
+ SELECT statement
+</code></pre>
+<p>Currently, there are 5 DataMap implementations in CarbonData.</p>
+<table>
+<thead>
+<tr>
+<th>DataMap Provider</th>
+<th>Description</th>
+<th>DMPROPERTIES</th>
+<th>Management</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>preaggregate</td>
+<td>single table pre-aggregate table</td>
+<td>No DMPROPERTY is required</td>
+<td>Automatic</td>
+</tr>
+<tr>
+<td>timeseries</td>
+<td>time dimension rollup table</td>
+<td>event_time, xx_granularity, please refer to <a href="./timeseries-datamap-guide.html">Timeseries DataMap</a>
+</td>
+<td>Automatic</td>
+</tr>
+<tr>
+<td>mv</td>
+<td>multi-table pre-aggregate table</td>
+<td>No DMPROPERTY is required</td>
+<td>Manual</td>
+</tr>
+<tr>
+<td>lucene</td>
+<td>lucene indexing for text column</td>
+<td>index_columns to specifying the index columns</td>
+<td>Automatic</td>
+</tr>
+<tr>
+<td>bloomfilter</td>
+<td>bloom filter for high cardinality column, geospatial column</td>
+<td>index_columns to specifying the index columns</td>
+<td>Automatic</td>
+</tr>
+</tbody>
+</table>
+<h2>
+<a id="datamap-management" class="anchor" href="#datamap-management" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>DataMap Management</h2>
+<p>There are two kinds of management semantic for DataMap.</p>
+<ol>
+<li>Automatic Refresh: Create datamap without <code>WITH DEFERRED REBUILD</code> in the statement, which is by default.</li>
+<li>Manual Refresh: Create datamap with <code>WITH DEFERRED REBUILD</code> in the statement</li>
+</ol>
+<p><strong>CAUTION:</strong>
+If user create MV datamap without specifying <code>WITH DEFERRED REBUILD</code>, carbondata will give a warning and treat the datamap as deferred rebuild.</p>
+<h3>
+<a id="automatic-refresh" class="anchor" href="#automatic-refresh" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Automatic Refresh</h3>
+<p>When user creates a datamap on the main table without using <code>WITH DEFERRED REBUILD</code> syntax, the datamap will be managed by system automatically.
+For every data load to the main table, system will immediately trigger a load to the datamap automatically. These two data loading (to main table and datamap) is executed in a transactional manner, meaning that it will be either both success or neither success.</p>
+<p>The data loading to datamap is incremental based on Segment concept, avoiding a expensive total rebuild.</p>
+<p>If user perform following command on the main table, system will return failure. (reject the operation)</p>
+<ol>
+<li>Data management command: <code>UPDATE/DELETE/DELETE SEGMENT</code>.</li>
+<li>Schema management command: <code>ALTER TABLE DROP COLUMN</code>, <code>ALTER TABLE CHANGE DATATYPE</code>,
+<code>ALTER TABLE RENAME</code>. Note that adding a new column is supported, and for dropping columns and
+change datatype command, CarbonData will check whether it will impact the pre-aggregate table, if
+not, the operation is allowed, otherwise operation will be rejected by throwing exception.</li>
+<li>Partition management command: `ALTER TABLE ADD/DROP PARTITION</li>
+</ol>
+<p>If user do want to perform above operations on the main table, user can first drop the datamap, perform the operation, and re-create the datamap again.</p>
+<p>If user drop the main table, the datamap will be dropped immediately too.</p>
+<p>We do recommend you to use this management for index datamap.</p>
+<h3>
+<a id="manual-refresh" class="anchor" href="#manual-refresh" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Manual Refresh</h3>
+<p>When user creates a datamap specifying manual refresh semantic, the datamap is created with status <em>disabled</em> and query will NOT use this datamap until user can issue REBUILD DATAMAP command to build the datamap. For every REBUILD DATAMAP command, system will trigger a full rebuild of the datamap. After rebuild is done, system will change datamap status to <em>enabled</em>, so that it can be used in query rewrite.</p>
+<p>For every new data loading, data update, delete, the related datamap will be made <em>disabled</em>,
+which means that the following queries will not benefit from the datamap before it becomes <em>enabled</em> again.</p>
+<p>If the main table is dropped by user, the related datamap will be dropped immediately.</p>
+<p><strong>Note</strong>:</p>
+<ul>
+<li>If you are creating a datamap on external table, you need to do manual management of the datamap.</li>
+<li>For index datamap such as BloomFilter datamap, there is no need to do manual refresh.
+By default it is automatic refresh,
+which means its data will get refreshed immediately after the datamap is created or the main table is loaded.
+Manual refresh on this datamap will has no impact.</li>
+</ul>
+<h2>
+<a id="datamap-catalog" class="anchor" href="#datamap-catalog" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>DataMap Catalog</h2>
+<p>Currently, when user creates a datamap, system will store the datamap metadata in a configurable <em>system</em> folder in HDFS or S3.</p>
+<p>In this <em>system</em> folder, it contains:</p>
+<ul>
+<li>DataMapSchema file. It is a json file containing schema for one datamap. Ses DataMapSchema class. If user creates 100 datamaps (on different tables), there will be 100 files in <em>system</em> folder.</li>
+<li>DataMapStatus file. Only one file, it is in json format, and each entry in the file represents for one datamap. Ses DataMapStatusDetail class</li>
+</ul>
+<p>There is a DataMapCatalog interface to retrieve schema of all datamap, it can be used in optimizer to get the metadata of datamap.</p>
+<h2>
+<a id="datamap-related-commands" class="anchor" href="#datamap-related-commands" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>DataMap Related Commands</h2>
+<h3>
+<a id="explain" class="anchor" href="#explain" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Explain</h3>
+<p>How can user know whether datamap is used in the query?</p>
+<p>User can set enable.query.statistics = true and use EXPLAIN command to know, it will print out something like</p>
+<pre lang="text"><code>== CarbonData Profiler ==
+Hit mv DataMap: datamap1
+Scan Table: default.datamap1_table
++- filter:
++- pruning by CG DataMap
++- all blocklets: 1
+ skipped blocklets: 0
+</code></pre>
+<h3>
+<a id="show-datamap" class="anchor" href="#show-datamap" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Show DataMap</h3>
+<p>There is a SHOW DATAMAPS command, when this is issued, system will read all datamap from <em>system</em> folder and print all information on screen. The current information includes:</p>
+<ul>
+<li>DataMapName</li>
+<li>DataMapProviderName like mv, preaggreagte, timeseries, etc</li>
+<li>Associated Table</li>
+</ul>
+<h3>
+<a id="compaction-on-datamap" class="anchor" href="#compaction-on-datamap" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Compaction on DataMap</h3>
+<p>This feature applies for preaggregate datamap only</p>
+<p>Running Compaction command (<code>ALTER TABLE COMPACT</code>) on main table will <strong>not automatically</strong> compact the pre-aggregate tables created on the main table. User need to run Compaction command separately on each pre-aggregate table to compact them.</p>
+<p>Compaction is an optional operation for pre-aggregate table. If compaction is performed on main table but not performed on pre-aggregate table, all queries still can benefit from pre-aggregate tables. To further improve the query performance, compaction on pre-aggregate tables can be triggered to merge the segments and files in the pre-aggregate tables.</p>
+<script>
+$(function() {
+ // Show selected style on nav item
+ $('.b-nav__datamap').addClass('selected');
+
+ if (!$('.b-nav__datamap').parent().hasClass('nav__item__with__subs--expanded')) {
+ // Display datamap subnav items
+ $('.b-nav__datamap').parent().toggleClass('nav__item__with__subs--expanded');
+ }
+});
+</script></div>
+</div>
+</div>
+</div>
+<div class="doc-footer">
+ <a href="#top" class="scroll-top">Top</a>
+</div>
+</div>
+</section>
+</div>
+</div>
+</div>
+</section><!-- End systemblock part -->
+<script src="js/custom.js"></script>
+</body>
+</html>
\ No newline at end of file
diff --git a/src/main/webapp/index.html b/src/main/webapp/index.html
index 636c117..7be4127 100644
--- a/src/main/webapp/index.html
+++ b/src/main/webapp/index.html
@@ -112,7 +112,6 @@
<li><a href="videogallery.html">CarbonData Videos </a></li>
</ul>
</li>
- <li><a href="documentation.html" class="">CN</a></li>
<li class="dropdown">
<a href="http://www.apache.org/"
class="apache_link hidden-xs dropdown-toggle" data-toggle="dropdown"
@@ -329,31 +328,31 @@
</h4>
<div class="linkblock">
<div class="block-row">
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
- target="_blank">Apache CarbonData 1.4.1</a>
- <span class="release-date">Aug 2018</span>
- <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Apache+CarbonData+1.4.1+Release"
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/2.1.1/"
+ target="_blank">Apache CarbonData 2.1.1</a>
+ <span class="release-date">May 2021</span>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Apache+CarbonData+2.1.1+Release"
class="whatsnew" target="_blank">what's new</a>
</div>
- <div class="block-row">
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
- target="_blank">Apache CarbonData 1.4.0</a>
- <span class="release-date">May 2018</span>
- <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Apache+CarbonData+1.4.0+Release"
+ <div class="block-row">
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/2.1.0/"
+ target="_blank">Apache CarbonData 2.1.0</a>
+ <span class="release-date">Nov 2020</span>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Apache+CarbonData+2.1.0+Relea"
class="whatsnew" target="_blank">what's new</a>
</div>
<div class="block-row">
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
- target="_blank">Apache CarbonData 1.3.1</a>
- <span class="release-date">Mar 2018</span>
- <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Apache+CarbonData+1.3.1+Release"
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/2.0.1/"
+ target="_blank">Apache CarbonData 2.0.1</a>
+ <span class="release-date">Jun 2020</span>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Apache+CarbonData+2.0.1+Release"
class="whatsnew" target="_blank">what's new</a>
</div>
<div class="block-row">
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.0/"
- target="_blank">Apache CarbonData 1.3.0</a>
- <span class="release-date">Feb 2018</span>
- <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Apache+CarbonData+1.3.0+Release"
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.6.1/"
+ target="_blank">Apache CarbonData 1.6.1</a>
+ <span class="release-date">Oct 2019</span>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Apache+CarbonData+1.6.1+Release"
class="whatsnew" target="_blank">what's new</a>
</div>
</div>
@@ -487,7 +486,7 @@
to do is:</p>
<ol class="orderlist">
<li>
- <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/"
target="_blank">Download</a>the latest release.
</li>
diff --git a/src/main/webapp/lucene-datamap-guide.html b/src/main/webapp/lucene-datamap-guide.html
new file mode 100644
index 0000000..f461ca5
--- /dev/null
+++ b/src/main/webapp/lucene-datamap-guide.html
@@ -0,0 +1,389 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href='images/favicon.ico' rel='shortcut icon' type='image/x-icon'>
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>CarbonData</title>
+ <style>
+
+ </style>
+ <!-- Bootstrap -->
+
+ <link rel="stylesheet" href="css/bootstrap.min.css">
+ <link href="css/style.css" rel="stylesheet">
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.scom/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ <script src="js/jquery.min.js"></script>
+ <script src="js/bootstrap.min.js"></script>
+ <script defer src="https://use.fontawesome.com/releases/v5.0.8/js/all.js"></script>
+
+
+</head>
+<body>
+<header>
+ <nav class="navbar navbar-default navbar-custom cd-navbar-wrapper">
+ <div class="container">
+ <div class="navbar-header">
+ <button aria-controls="navbar" aria-expanded="false" data-target="#navbar" data-toggle="collapse"
+ class="navbar-toggle collapsed" type="button">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a href="index.html" class="logo">
+ <img src="images/CarbonDataLogo.png" alt="CarbonData logo" title="CarbocnData logo"/>
+ </a>
+ </div>
+ <div class="navbar-collapse collapse cd_navcontnt" id="navbar">
+ <ul class="nav navbar-nav navbar-right navlist-custom">
+ <li><a href="index.html" class="hidden-xs"><i class="fa fa-home" aria-hidden="true"></i> </a>
+ </li>
+ <li><a href="index.html" class="hidden-lg hidden-md hidden-sm">Home</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle " data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false"> Download <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.0/"
+ target="_blank">Apache CarbonData 1.5.0</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
+ target="_blank">Apache CarbonData 1.4.1</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
+ target="_blank">Apache CarbonData 1.4.0</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
+ target="_blank">Apache CarbonData 1.3.1</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.0/"
+ target="_blank">Apache CarbonData 1.3.0</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
+ target="_blank">Release Archive</a></li>
+ </ul>
+ </li>
+ <li><a href="documentation.html" class="active">Documentation</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false">Community <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://github.com/apache/carbondata/blob/master/docs/how-to-contribute-to-apache-carbondata.md"
+ target="_blank">Contributing to CarbonData</a></li>
+ <li>
+ <a href="https://github.com/apache/carbondata/blob/master/docs/release-guide.md"
+ target="_blank">Release Guide</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/PMC+and+Committers+member+list"
+ target="_blank">Project PMC and Committers</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=66850609"
+ target="_blank">CarbonData Meetups</a></li>
+ <li><a href="security.html">Apache CarbonData Security</a></li>
+ <li><a href="https://issues.apache.org/jira/browse/CARBONDATA" target="_blank">Apache
+ Jira</a></li>
+ <li><a href="videogallery.html">CarbonData Videos </a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="apache_link hidden-xs dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="hidden-lg hidden-md hidden-sm dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+
+ <li>
+ <a href="#" id="search-icon"><i class="fa fa-search" aria-hidden="true"></i></a>
+
+ </li>
+
+ </ul>
+ </div><!--/.nav-collapse -->
+ <div id="search-box">
+ <form method="get" action="http://www.google.com/search" target="_blank">
+ <div class="search-block">
+ <table border="0" cellpadding="0" width="100%">
+ <tr>
+ <td style="width:80%">
+ <input type="text" name="q" size=" 5" maxlength="255" value=""
+ class="search-input" placeholder="Search...." required/>
+ </td>
+ <td style="width:20%">
+ <input type="submit" value="Search"/></td>
+ </tr>
+ <tr>
+ <td align="left" style="font-size:75%" colspan="2">
+ <input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
+ </td>
+ </tr>
+ </table>
+ </div>
+ </form>
+ </div>
+ </div>
+ </nav>
+</header> <!-- end Header part -->
+
+<div class="fixed-padding"></div> <!-- top padding with fixde header -->
+
+<section><!-- Dashboard nav -->
+ <div class="container-fluid q">
+ <div class="col-sm-12 col-md-12 maindashboard">
+ <div class="verticalnavbar">
+ <nav class="b-sticky-nav">
+ <div class="nav-scroller">
+ <div class="nav__inner">
+ <a class="b-nav__intro nav__item" href="./introduction.html">introduction</a>
+ <a class="b-nav__quickstart nav__item" href="./quick-start-guide.html">quick start</a>
+ <a class="b-nav__uses nav__item" href="./usecases.html">use cases</a>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__docs nav__item nav__sub__anchor" href="./language-manual.html">Language Reference</a>
+ <a class="nav__item nav__sub__item" href="./ddl-of-carbondata.html">DDL</a>
+ <a class="nav__item nav__sub__item" href="./dml-of-carbondata.html">DML</a>
+ <a class="nav__item nav__sub__item" href="./streaming-guide.html">Streaming</a>
+ <a class="nav__item nav__sub__item" href="./configuration-parameters.html">Configuration</a>
+ <a class="nav__item nav__sub__item" href="./datamap-developer-guide.html">Datamaps</a>
+ <a class="nav__item nav__sub__item" href="./supported-data-types-in-carbondata.html">Data Types</a>
+ </div>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__datamap nav__item nav__sub__anchor" href="./datamap-management.html">DataMaps</a>
+ <a class="nav__item nav__sub__item" href="./bloomfilter-datamap-guide.html">Bloom Filter</a>
+ <a class="nav__item nav__sub__item" href="./lucene-datamap-guide.html">Lucene</a>
+ <a class="nav__item nav__sub__item" href="./preaggregate-datamap-guide.html">Pre-Aggregate</a>
+ <a class="nav__item nav__sub__item" href="./timeseries-datamap-guide.html">Time Series</a>
+ </div>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__api nav__item nav__sub__anchor" href="./sdk-guide.html">API</a>
+ <a class="nav__item nav__sub__item" href="./sdk-guide.html">Java SDK</a>
+ <a class="nav__item nav__sub__item" href="./CSDK-guide.html">C++ SDK</a>
+ </div>
+
+ <a class="b-nav__perf nav__item" href="./performance-tuning.html">Performance Tuning</a>
+ <a class="b-nav__s3 nav__item" href="./s3-guide.html">S3 Storage</a>
+ <a class="b-nav__faq nav__item" href="./faq.html">FAQ</a>
+ <a class="b-nav__contri nav__item" href="./how-to-contribute-to-apache-carbondata.html">Contribute</a>
+ <a class="b-nav__security nav__item" href="./security.html">Security</a>
+ <a class="b-nav__release nav__item" href="./release-guide.html">Release Guide</a>
+ </div>
+ </div>
+ <div class="navindicator">
+ <div class="b-nav__intro navindicator__item"></div>
+ <div class="b-nav__quickstart navindicator__item"></div>
+ <div class="b-nav__uses navindicator__item"></div>
+ <div class="b-nav__docs navindicator__item"></div>
+ <div class="b-nav__datamap navindicator__item"></div>
+ <div class="b-nav__api navindicator__item"></div>
+ <div class="b-nav__perf navindicator__item"></div>
+ <div class="b-nav__s3 navindicator__item"></div>
+ <div class="b-nav__faq navindicator__item"></div>
+ <div class="b-nav__contri navindicator__item"></div>
+ <div class="b-nav__security navindicator__item"></div>
+ </div>
+ </nav>
+ </div>
+ <div class="mdcontent">
+ <section>
+ <div style="padding:10px 15px;">
+ <div id="viewpage" name="viewpage">
+ <div class="row">
+ <div class="col-sm-12 col-md-12">
+ <div>
+<h1>
+<a id="carbondata-lucene-datamap-alpha-feature" class="anchor" href="#carbondata-lucene-datamap-alpha-feature" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CarbonData Lucene DataMap (Alpha Feature)</h1>
+<ul>
+<li><a href="#datamap-management">DataMap Management</a></li>
+<li><a href="#lucene-datamap-introduction">Lucene Datamap</a></li>
+<li><a href="#loading-data">Loading Data</a></li>
+<li><a href="#querying-data">Querying Data</a></li>
+<li><a href="#data-management-with-lucene-datamap">Data Management</a></li>
+</ul>
+<h4>
+<a id="datamap-management" class="anchor" href="#datamap-management" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>DataMap Management</h4>
+<p>Lucene DataMap can be created using following DDL</p>
+<pre><code>CREATE DATAMAP [IF NOT EXISTS] datamap_name
+ON TABLE main_table
+USING 'lucene'
+DMPROPERTIES ('index_columns'='city, name', ...)
+</code></pre>
+<p>DataMap can be dropped using following DDL:</p>
+<pre><code>DROP DATAMAP [IF EXISTS] datamap_name
+ON TABLE main_table
+</code></pre>
+<p>To show all DataMaps created, use:</p>
+<pre><code>SHOW DATAMAP
+ON TABLE main_table
+</code></pre>
+<p>It will show all DataMaps created on main table.</p>
+<h2>
+<a id="lucene-datamap-introduction" class="anchor" href="#lucene-datamap-introduction" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Lucene DataMap Introduction</h2>
+<p>Lucene is a high performance, full featured text search engine. Lucene is integrated to carbon as
+an index datamap and managed along with main tables by CarbonData. User can create lucene datamap
+to improve query performance on string columns which has content of more length. So, user can
+search tokenized word or pattern of it using lucene query on text content.</p>
+<p>For instance, main table called <strong>datamap_test</strong> which is defined as:</p>
+<pre><code>CREATE TABLE datamap_test (
+ name string,
+ age int,
+ city string,
+ country string)
+STORED AS carbondata
+</code></pre>
+<p>User can create Lucene datamap using the Create DataMap DDL:</p>
+<pre><code>CREATE DATAMAP dm
+ON TABLE datamap_test
+USING 'lucene'
+DMPROPERTIES ('INDEX_COLUMNS' = 'name, country',)
+</code></pre>
+<p><strong>DMProperties</strong></p>
+<ol>
+<li>INDEX_COLUMNS: The list of string columns on which lucene creates indexes.</li>
+<li>FLUSH_CACHE: size of the cache to maintain in Lucene writer, if specified then it tries to
+aggregate the unique data till the cache limit and flush to Lucene. It is best suitable for low
+cardinality dimensions.</li>
+<li>SPLIT_BLOCKLET: when made as true then store the data in blocklet wise in lucene , it means new
+folder will be created for each blocklet, thus, it eliminates storing blockletid in lucene and
+also it makes lucene small chunks of data.</li>
+</ol>
+<h2>
+<a id="loading-data" class="anchor" href="#loading-data" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Loading data</h2>
+<p>When loading data to main table, lucene index files will be generated for all the
+index_columns(String Columns) given in DMProperties which contains information about the data
+location of index_columns. These index files will be written inside a folder named with datamap name
+inside each segment folders.</p>
+<p>A system level configuration carbon.lucene.compression.mode can be added for best compression of
+lucene index files. The default value is speed, where the index writing speed will be more. If the
+value is compression, the index file size will be compressed.</p>
+<h2>
+<a id="querying-data" class="anchor" href="#querying-data" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Querying data</h2>
+<p>As a technique for query acceleration, Lucene indexes cannot be queried directly.
+Queries are to be made on main table. when a query with TEXT_MATCH('name:c10') or
+TEXT_MATCH_WITH_LIMIT('name:n10',10)[the second parameter represents the number of result to be
+returned, if user does not specify this value, all results will be returned without any limit] is
+fired, two jobs are fired. The first job writes the temporary files in folder created at table level
+which contains lucene's seach results and these files will be read in second job to give faster
+results. These temporary files will be cleared once the query finishes.</p>
+<p>User can verify whether a query can leverage Lucene datamap or not by executing <code>EXPLAIN</code>
+command, which will show the transformed logical plan, and thus user can check whether TEXT_MATCH()
+filter is applied on query or not.</p>
+<p><strong>Note:</strong></p>
+<ol>
+<li>
+<p>The filter columns in TEXT_MATCH or TEXT_MATCH_WITH_LIMIT must be always in lower case and
+filter condition like 'AND','OR' must be in upper case.</p>
+<p>Ex:</p>
+<pre><code>select * from datamap_test where TEXT_MATCH('name:*10 AND name:*n*')
+</code></pre>
+</li>
+<li>
+<p>Query supports only one TEXT_MATCH udf for filter condition and not multiple udfs.</p>
+<p>The following query is supported:</p>
+<pre><code>select * from datamap_test where TEXT_MATCH('name:*10 AND name:*n*')
+</code></pre>
+<p>The following query is not supported:</p>
+<pre><code>select * from datamap_test where TEXT_MATCH('name:*10) AND TEXT_MATCH(name:*n*')
+</code></pre>
+</li>
+</ol>
+<p>Below like queries can be converted to text_match queries as following:</p>
+<pre><code>select * from datamap_test where name='n10'
+
+select * from datamap_test where name like 'n1%'
+
+select * from datamap_test where name like '%10'
+
+select * from datamap_test where name like '%n%'
+
+select * from datamap_test where name like '%10' and name not like '%n%'
+</code></pre>
+<p>Lucene TEXT_MATCH Queries:</p>
+<pre><code>select * from datamap_test where TEXT_MATCH('name:n10')
+
+select * from datamap_test where TEXT_MATCH('name:n1*')
+
+select * from datamap_test where TEXT_MATCH('name:*10')
+
+select * from datamap_test where TEXT_MATCH('name:*n*')
+
+select * from datamap_test where TEXT_MATCH('name:*10 -name:*n*')
+</code></pre>
+<p><strong>Note:</strong> For lucene queries and syntax, refer to <a href="http://www.lucenetutorial.com/lucene-query-syntax.html" target=_blank rel="nofollow">lucene-syntax</a></p>
+<h2>
+<a id="data-management-with-lucene-datamap" class="anchor" href="#data-management-with-lucene-datamap" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Data Management with lucene datamap</h2>
+<p>Once there is lucene datamap is created on the main table, following command on the main
+table
+is not supported:</p>
+<ol>
+<li>Data management command: <code>UPDATE/DELETE</code>.</li>
+<li>Schema management command: <code>ALTER TABLE DROP COLUMN</code>, <code>ALTER TABLE CHANGE DATATYPE</code>,
+<code>ALTER TABLE RENAME</code>.</li>
+</ol>
+<p><strong>Note</strong>: Adding a new column is supported, and for dropping columns and change datatype
+command, CarbonData will check whether it will impact the lucene datamap, if not, the operation
+is allowed, otherwise operation will be rejected by throwing exception.</p>
+<ol start="3">
+<li>Partition management command: <code>ALTER TABLE ADD/DROP PARTITION</code>.</li>
+</ol>
+<p>However, there is still way to support these operations on main table, in current CarbonData
+release, user can do as following:</p>
+<ol>
+<li>Remove the lucene datamap by <code>DROP DATAMAP</code> command.</li>
+<li>Carry out the data management operation on main table.</li>
+<li>Create the lucene datamap again by <code>CREATE DATAMAP</code> command.
+Basically, user can manually trigger the operation by re-building the datamap.</li>
+</ol>
+<script>
+$(function() {
+ // Show selected style on nav item
+ $('.b-nav__datamap').addClass('selected');
+
+ if (!$('.b-nav__datamap').parent().hasClass('nav__item__with__subs--expanded')) {
+ // Display datamap subnav items
+ $('.b-nav__datamap').parent().toggleClass('nav__item__with__subs--expanded');
+ }
+});
+</script></div>
+</div>
+</div>
+</div>
+<div class="doc-footer">
+ <a href="#top" class="scroll-top">Top</a>
+</div>
+</div>
+</section>
+</div>
+</div>
+</div>
+</section><!-- End systemblock part -->
+<script src="js/custom.js"></script>
+</body>
+</html>
\ No newline at end of file
diff --git a/src/main/webapp/pdf/TPCDS_Report_CarbonData2.1.0.pdf b/src/main/webapp/pdf/TPCDS_Report_CarbonData2.1.0.pdf
new file mode 100644
index 0000000..1c3ac79
Binary files /dev/null and b/src/main/webapp/pdf/TPCDS_Report_CarbonData2.1.0.pdf differ
diff --git a/src/main/webapp/preaggregate-datamap-guide.html b/src/main/webapp/preaggregate-datamap-guide.html
new file mode 100644
index 0000000..e4f1c91
--- /dev/null
+++ b/src/main/webapp/preaggregate-datamap-guide.html
@@ -0,0 +1,502 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href='images/favicon.ico' rel='shortcut icon' type='image/x-icon'>
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>CarbonData</title>
+ <style>
+
+ </style>
+ <!-- Bootstrap -->
+
+ <link rel="stylesheet" href="css/bootstrap.min.css">
+ <link href="css/style.css" rel="stylesheet">
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.scom/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ <script src="js/jquery.min.js"></script>
+ <script src="js/bootstrap.min.js"></script>
+ <script defer src="https://use.fontawesome.com/releases/v5.0.8/js/all.js"></script>
+
+
+</head>
+<body>
+<header>
+ <nav class="navbar navbar-default navbar-custom cd-navbar-wrapper">
+ <div class="container">
+ <div class="navbar-header">
+ <button aria-controls="navbar" aria-expanded="false" data-target="#navbar" data-toggle="collapse"
+ class="navbar-toggle collapsed" type="button">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a href="index.html" class="logo">
+ <img src="images/CarbonDataLogo.png" alt="CarbonData logo" title="CarbocnData logo"/>
+ </a>
+ </div>
+ <div class="navbar-collapse collapse cd_navcontnt" id="navbar">
+ <ul class="nav navbar-nav navbar-right navlist-custom">
+ <li><a href="index.html" class="hidden-xs"><i class="fa fa-home" aria-hidden="true"></i> </a>
+ </li>
+ <li><a href="index.html" class="hidden-lg hidden-md hidden-sm">Home</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle " data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false"> Download <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.0/"
+ target="_blank">Apache CarbonData 1.5.0</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
+ target="_blank">Apache CarbonData 1.4.1</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
+ target="_blank">Apache CarbonData 1.4.0</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
+ target="_blank">Apache CarbonData 1.3.1</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.0/"
+ target="_blank">Apache CarbonData 1.3.0</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
+ target="_blank">Release Archive</a></li>
+ </ul>
+ </li>
+ <li><a href="documentation.html" class="active">Documentation</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false">Community <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://github.com/apache/carbondata/blob/master/docs/how-to-contribute-to-apache-carbondata.md"
+ target="_blank">Contributing to CarbonData</a></li>
+ <li>
+ <a href="https://github.com/apache/carbondata/blob/master/docs/release-guide.md"
+ target="_blank">Release Guide</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/PMC+and+Committers+member+list"
+ target="_blank">Project PMC and Committers</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=66850609"
+ target="_blank">CarbonData Meetups</a></li>
+ <li><a href="security.html">Apache CarbonData Security</a></li>
+ <li><a href="https://issues.apache.org/jira/browse/CARBONDATA" target="_blank">Apache
+ Jira</a></li>
+ <li><a href="videogallery.html">CarbonData Videos </a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="apache_link hidden-xs dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="hidden-lg hidden-md hidden-sm dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+
+ <li>
+ <a href="#" id="search-icon"><i class="fa fa-search" aria-hidden="true"></i></a>
+
+ </li>
+
+ </ul>
+ </div><!--/.nav-collapse -->
+ <div id="search-box">
+ <form method="get" action="http://www.google.com/search" target="_blank">
+ <div class="search-block">
+ <table border="0" cellpadding="0" width="100%">
+ <tr>
+ <td style="width:80%">
+ <input type="text" name="q" size=" 5" maxlength="255" value=""
+ class="search-input" placeholder="Search...." required/>
+ </td>
+ <td style="width:20%">
+ <input type="submit" value="Search"/></td>
+ </tr>
+ <tr>
+ <td align="left" style="font-size:75%" colspan="2">
+ <input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
+ </td>
+ </tr>
+ </table>
+ </div>
+ </form>
+ </div>
+ </div>
+ </nav>
+</header> <!-- end Header part -->
+
+<div class="fixed-padding"></div> <!-- top padding with fixde header -->
+
+<section><!-- Dashboard nav -->
+ <div class="container-fluid q">
+ <div class="col-sm-12 col-md-12 maindashboard">
+ <div class="verticalnavbar">
+ <nav class="b-sticky-nav">
+ <div class="nav-scroller">
+ <div class="nav__inner">
+ <a class="b-nav__intro nav__item" href="./introduction.html">introduction</a>
+ <a class="b-nav__quickstart nav__item" href="./quick-start-guide.html">quick start</a>
+ <a class="b-nav__uses nav__item" href="./usecases.html">use cases</a>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__docs nav__item nav__sub__anchor" href="./language-manual.html">Language Reference</a>
+ <a class="nav__item nav__sub__item" href="./ddl-of-carbondata.html">DDL</a>
+ <a class="nav__item nav__sub__item" href="./dml-of-carbondata.html">DML</a>
+ <a class="nav__item nav__sub__item" href="./streaming-guide.html">Streaming</a>
+ <a class="nav__item nav__sub__item" href="./configuration-parameters.html">Configuration</a>
+ <a class="nav__item nav__sub__item" href="./datamap-developer-guide.html">Datamaps</a>
+ <a class="nav__item nav__sub__item" href="./supported-data-types-in-carbondata.html">Data Types</a>
+ </div>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__datamap nav__item nav__sub__anchor" href="./datamap-management.html">DataMaps</a>
+ <a class="nav__item nav__sub__item" href="./bloomfilter-datamap-guide.html">Bloom Filter</a>
+ <a class="nav__item nav__sub__item" href="./lucene-datamap-guide.html">Lucene</a>
+ <a class="nav__item nav__sub__item" href="./preaggregate-datamap-guide.html">Pre-Aggregate</a>
+ <a class="nav__item nav__sub__item" href="./timeseries-datamap-guide.html">Time Series</a>
+ </div>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__api nav__item nav__sub__anchor" href="./sdk-guide.html">API</a>
+ <a class="nav__item nav__sub__item" href="./sdk-guide.html">Java SDK</a>
+ <a class="nav__item nav__sub__item" href="./CSDK-guide.html">C++ SDK</a>
+ </div>
+
+ <a class="b-nav__perf nav__item" href="./performance-tuning.html">Performance Tuning</a>
+ <a class="b-nav__s3 nav__item" href="./s3-guide.html">S3 Storage</a>
+ <a class="b-nav__faq nav__item" href="./faq.html">FAQ</a>
+ <a class="b-nav__contri nav__item" href="./how-to-contribute-to-apache-carbondata.html">Contribute</a>
+ <a class="b-nav__security nav__item" href="./security.html">Security</a>
+ <a class="b-nav__release nav__item" href="./release-guide.html">Release Guide</a>
+ </div>
+ </div>
+ <div class="navindicator">
+ <div class="b-nav__intro navindicator__item"></div>
+ <div class="b-nav__quickstart navindicator__item"></div>
+ <div class="b-nav__uses navindicator__item"></div>
+ <div class="b-nav__docs navindicator__item"></div>
+ <div class="b-nav__datamap navindicator__item"></div>
+ <div class="b-nav__api navindicator__item"></div>
+ <div class="b-nav__perf navindicator__item"></div>
+ <div class="b-nav__s3 navindicator__item"></div>
+ <div class="b-nav__faq navindicator__item"></div>
+ <div class="b-nav__contri navindicator__item"></div>
+ <div class="b-nav__security navindicator__item"></div>
+ </div>
+ </nav>
+ </div>
+ <div class="mdcontent">
+ <section>
+ <div style="padding:10px 15px;">
+ <div id="viewpage" name="viewpage">
+ <div class="row">
+ <div class="col-sm-12 col-md-12">
+ <div>
+<h1>
+<a id="carbondata-pre-aggregate-datamap" class="anchor" href="#carbondata-pre-aggregate-datamap" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CarbonData Pre-aggregate DataMap</h1>
+<ul>
+<li><a href="#quick-example">Quick Example</a></li>
+<li><a href="#datamap-management">DataMap Management</a></li>
+<li><a href="#preaggregate-datamap-introduction">Pre-aggregate Table</a></li>
+<li><a href="#loading-data">Loading Data</a></li>
+<li><a href="#querying-data">Querying Data</a></li>
+<li><a href="#compacting-pre-aggregate-tables">Compaction</a></li>
+<li><a href="#data-management-with-pre-aggregate-tables">Data Management</a></li>
+</ul>
+<h2>
+<a id="quick-example" class="anchor" href="#quick-example" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Quick example</h2>
+<p>Download and unzip spark-2.2.0-bin-hadoop2.7.tgz, and export $SPARK_HOME</p>
+<p>Package carbon jar, and copy assembly/target/scala-2.11/carbondata_2.11-x.x.x-SNAPSHOT-shade-hadoop2.7.2.jar to $SPARK_HOME/jars</p>
+<div class="highlight highlight-source-shell"><pre>mvn clean package -DskipTests -Pspark-2.2</pre></div>
+<p>Start spark-shell in new terminal, type :paste, then copy and run the following code.</p>
+<div class="highlight highlight-source-scala"><pre> <span class="pl-k">import</span> <span class="pl-en">java</span>.<span class="pl-en">io</span>.<span class="pl-en">File</span>
+ <span class="pl-k">import</span> <span class="pl-en">org</span>.<span class="pl-en">apache</span>.<span class="pl-en">spark</span>.<span class="pl-en">sql</span>.{<span class="pl-en">CarbonEnv</span>, <span class="pl-en">SparkSession</span>}
+ <span class="pl-k">import</span> <span class="pl-en">org</span>.<span class="pl-en">apache</span>.<span class="pl-en">spark</span>.<span class="pl-en">sql</span>.<span class="pl-en">CarbonSession</span>.<span class="pl-en">_</span>
+ <span class="pl-k">import</span> <span class="pl-en">org</span>.<span class="pl-en">apache</span>.<span class="pl-en">spark</span>.<span class="pl-en">sql</span>.<span class="pl-en">streaming</span>.{<span class="pl-en">ProcessingTime</span>, <span class="pl-en">StreamingQuery</span>}
+ <span class="pl-k">import</span> <span class="pl-en">org</span>.<span class="pl-en">apache</span>.<span class="pl-en">carbondata</span>.<span class="pl-en">core</span>.<span class="pl-en">util</span>.<span class="pl-en">path</span>.<span class="pl-en">CarbonStorePath</span>
+
+ <span class="pl-k">val</span> <span class="pl-smi">warehouse</span> <span class="pl-k">=</span> <span class="pl-k">new</span> <span class="pl-en">File</span>(<span class="pl-s"><span class="pl-pds">"</span>./warehouse<span class="pl-pds">"</span></span>).getCanonicalPath
+ <span class="pl-k">val</span> <span class="pl-smi">metastore</span> <span class="pl-k">=</span> <span class="pl-k">new</span> <span class="pl-en">File</span>(<span class="pl-s"><span class="pl-pds">"</span>./metastore<span class="pl-pds">"</span></span>).getCanonicalPath
+
+ <span class="pl-k">val</span> <span class="pl-smi">spark</span> <span class="pl-k">=</span> <span class="pl-en">SparkSession</span>
+ .builder()
+ .master(<span class="pl-s"><span class="pl-pds">"</span>local<span class="pl-pds">"</span></span>)
+ .appName(<span class="pl-s"><span class="pl-pds">"</span>preAggregateExample<span class="pl-pds">"</span></span>)
+ .config(<span class="pl-s"><span class="pl-pds">"</span>spark.sql.warehouse.dir<span class="pl-pds">"</span></span>, warehouse)
+ .getOrCreateCarbonSession(warehouse, metastore)
+
+ spark.sparkContext.setLogLevel(<span class="pl-s"><span class="pl-pds">"</span>ERROR<span class="pl-pds">"</span></span>)
+
+ <span class="pl-c"><span class="pl-c">//</span> drop table if exists previously</span>
+ spark.sql(s<span class="pl-s"><span class="pl-pds">"</span>DROP TABLE IF EXISTS sales<span class="pl-pds">"</span></span>)
+
+ <span class="pl-c"><span class="pl-c">//</span> Create main table</span>
+ spark.sql(
+ s<span class="pl-s"><span class="pl-pds">"""</span></span>
+<span class="pl-s"> | CREATE TABLE sales (</span>
+<span class="pl-s"> | user_id string,</span>
+<span class="pl-s"> | country string,</span>
+<span class="pl-s"> | quantity int,</span>
+<span class="pl-s"> | price bigint)</span>
+<span class="pl-s"> | STORED AS carbondata</span>
+<span class="pl-s"> <span class="pl-pds">"""</span></span>.stripMargin)
+
+ <span class="pl-c"><span class="pl-c">//</span> Create pre-aggregate table on the main table</span>
+ <span class="pl-c"><span class="pl-c">//</span> If main table already have data, following command </span>
+ <span class="pl-c"><span class="pl-c">//</span> will trigger one immediate load to the pre-aggregate table</span>
+ spark.sql(
+ s<span class="pl-s"><span class="pl-pds">"""</span></span>
+<span class="pl-s"> | CREATE DATAMAP agg_sales</span>
+<span class="pl-s"> | ON TABLE sales</span>
+<span class="pl-s"> | USING "preaggregate"</span>
+<span class="pl-s"> | AS</span>
+<span class="pl-s"> | SELECT country, sum(quantity), avg(price)</span>
+<span class="pl-s"> | FROM sales</span>
+<span class="pl-s"> | GROUP BY country</span>
+<span class="pl-s"> <span class="pl-pds">"""</span></span>.stripMargin)
+
+ <span class="pl-k">import</span> <span class="pl-en">spark</span>.<span class="pl-en">implicits</span>.<span class="pl-en">_</span>
+ <span class="pl-k">import</span> <span class="pl-en">org</span>.<span class="pl-en">apache</span>.<span class="pl-en">spark</span>.<span class="pl-en">sql</span>.<span class="pl-en">SaveMode</span>
+ <span class="pl-k">import</span> <span class="pl-en">scala</span>.<span class="pl-en">util</span>.<span class="pl-en">Random</span>
+
+ <span class="pl-c"><span class="pl-c">//</span> Load data to the main table, it will also</span>
+ <span class="pl-c"><span class="pl-c">//</span> trigger immediate load to pre-aggregate table.</span>
+ <span class="pl-c"><span class="pl-c">//</span> These two loading operation is carried out in a</span>
+ <span class="pl-c"><span class="pl-c">//</span> transactional manner, meaning that the whole </span>
+ <span class="pl-c"><span class="pl-c">//</span> operation will fail if one of the loading fails</span>
+ <span class="pl-k">val</span> <span class="pl-smi">r</span> <span class="pl-k">=</span> <span class="pl-k">new</span> <span class="pl-en">Random</span>()
+ spark.sparkContext.parallelize(<span class="pl-c1">1</span> to <span class="pl-c1">10</span>)
+ .map(x <span class="pl-k">=></span> (<span class="pl-s"><span class="pl-pds">"</span>ID.<span class="pl-pds">"</span></span> <span class="pl-k">+</span> r.nextInt(<span class="pl-c1">100000</span>), <span class="pl-s"><span class="pl-pds">"</span>country<span class="pl-pds">"</span></span> <span class="pl-k">+</span> x <span class="pl-k">%</span> <span class="pl-c1">8</span>, x <span class="pl-k">%</span> <span class="pl-c1">50</span>, x <span class="pl-k">%</span> <span class="pl- [...]
+ .toDF(<span class="pl-s"><span class="pl-pds">"</span>user_id<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>country<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>quantity<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>price<span class="pl-pds">"</span></span>)
+ .write
+ .format(<span class="pl-s"><span class="pl-pds">"</span>carbondata<span class="pl-pds">"</span></span>)
+ .option(<span class="pl-s"><span class="pl-pds">"</span>tableName<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>sales<span class="pl-pds">"</span></span>)
+ .option(<span class="pl-s"><span class="pl-pds">"</span>compress<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>true<span class="pl-pds">"</span></span>)
+ .mode(<span class="pl-en">SaveMode</span>.<span class="pl-en">Append</span>)
+ .save()
+
+ spark.sql(
+ s<span class="pl-s"><span class="pl-pds">"""</span></span>
+<span class="pl-s"> |SELECT country, sum(quantity), avg(price)</span>
+<span class="pl-s"> | from sales GROUP BY country</span>
+<span class="pl-s"> <span class="pl-pds">"""</span></span>.stripMargin).show
+
+ spark.stop</pre></div>
+<h4>
+<a id="datamap-management" class="anchor" href="#datamap-management" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>DataMap Management</h4>
+<p>DataMap can be created using following DDL</p>
+<pre><code>CREATE DATAMAP [IF NOT EXISTS] datamap_name
+ON TABLE main_table
+USING "datamap_provider"
+DMPROPERTIES ('key'='value', ...)
+AS
+ SELECT statement
+</code></pre>
+<p>The string followed by USING is called DataMap Provider, in this version CarbonData supports two
+kinds of DataMap:</p>
+<ol>
+<li>preaggregate, for pre-aggregate table. Pre-Aggregate table supports two values for DMPROPERTIES.
+a. 'path' is used to specify the store location of the datamap.('path'='/location/').
+b. 'partitioning' when set to false enables user to disable partitioning of the datamap.
+Default value is true for this property.</li>
+<li>timeseries, for timeseries roll-up table. Please refer to <a href="./timeseries-datamap-guide.html">Timeseries DataMap</a>
+</li>
+</ol>
+<p>DataMap can be dropped using following DDL</p>
+<pre><code>DROP DATAMAP [IF EXISTS] datamap_name
+ON TABLE main_table
+</code></pre>
+<p>To show all DataMaps created, use:</p>
+<pre><code>SHOW DATAMAP
+ON TABLE main_table
+</code></pre>
+<p>It will show all DataMaps created on main table.</p>
+<h2>
+<a id="preaggregate-datamap-introduction" class="anchor" href="#preaggregate-datamap-introduction" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Preaggregate DataMap Introduction</h2>
+<p>Pre-aggregate tables are created as DataMaps and managed as tables internally by CarbonData.
+User can create as many pre-aggregate datamaps required to improve query performance,
+provided the storage requirements and loading speeds are acceptable.</p>
+<p>Once pre-aggregate datamaps are created, CarbonData's SparkSQL optimizer extension supports to
+select the most efficient pre-aggregate datamap and rewrite the SQL to query against the selected
+datamap instead of the main table. Since the data size of pre-aggregate datamap is smaller,
+user queries are much faster. In our previous experience, we have seen 5X to 100X times faster
+in production SQLs.</p>
+<p>For instance, main table called <strong>sales</strong> which is defined as</p>
+<pre><code>CREATE TABLE sales (
+ order_time timestamp,
+ user_id string,
+ sex string,
+ country string,
+ quantity int,
+ price bigint)
+STORED AS carbondata
+</code></pre>
+<p>User can create pre-aggregate tables using the Create DataMap DDL</p>
+<pre><code>CREATE DATAMAP agg_sales
+ON TABLE sales
+USING "preaggregate"
+AS
+ SELECT country, sex, sum(quantity), avg(price)
+ FROM sales
+ GROUP BY country, sex
+</code></pre>
+<h4>
+<a id="functions-supported-in-pre-aggregate-table" class="anchor" href="#functions-supported-in-pre-aggregate-table" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Functions supported in pre-aggregate table</h4>
+<table>
+<thead>
+<tr>
+<th>Function</th>
+<th align="center">Rollup supported</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>SUM</td>
+<td align="center">Yes</td>
+</tr>
+<tr>
+<td>AVG</td>
+<td align="center">Yes</td>
+</tr>
+<tr>
+<td>MAX</td>
+<td align="center">Yes</td>
+</tr>
+<tr>
+<td>MIN</td>
+<td align="center">Yes</td>
+</tr>
+<tr>
+<td>COUNT</td>
+<td align="center">Yes</td>
+</tr>
+</tbody>
+</table>
+<h4>
+<a id="how-pre-aggregate-tables-are-selected" class="anchor" href="#how-pre-aggregate-tables-are-selected" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>How pre-aggregate tables are selected</h4>
+<p>When a user query is submitted, during query planning phase, CarbonData will collect all matched
+pre-aggregate tables as candidates according to Relational Algebra transformation rules. Then, the
+best pre-aggregate table for this query will be selected among the candidates based on cost.
+For simplicity, current cost estimation is based on the data size of the pre-aggregate table. (We
+assume that query will be faster on smaller table)</p>
+<p>For the main table <strong>sales</strong> and pre-aggregate table <strong>agg_sales</strong> created above, following queries</p>
+<pre><code>SELECT country, sex, sum(quantity), avg(price) from sales GROUP BY country, sex
+
+SELECT sex, sum(quantity) from sales GROUP BY sex
+
+SELECT avg(price), country from sales GROUP BY country
+</code></pre>
+<p>will be transformed by CarbonData's query planner to query against pre-aggregate table
+<strong>agg_sales</strong> instead of the main table <strong>sales</strong></p>
+<p>However, for following queries</p>
+<pre><code>SELECT user_id, country, sex, sum(quantity), avg(price) from sales GROUP BY user_id, country, sex
+
+SELECT sex, avg(quantity) from sales GROUP BY sex
+
+SELECT country, max(price) from sales GROUP BY country
+</code></pre>
+<p>will query against main table <strong>sales</strong> only, because it does not satisfy pre-aggregate table
+selection logic.</p>
+<h2>
+<a id="loading-data" class="anchor" href="#loading-data" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Loading data</h2>
+<p>For existing table with loaded data, data load to pre-aggregate table will be triggered by the
+CREATE DATAMAP statement when user creates the pre-aggregate table. For incremental loads after
+aggregates tables are created, loading data to main table triggers the load to pre-aggregate tables
+once main table loading is complete.</p>
+<p>These loads are transactional
+meaning that data on main table and pre-aggregate tables are only visible to the user after all
+tables are loaded successfully, if one of these loads fails, new data are not visible in all tables
+as if the load operation is not happened.</p>
+<h2>
+<a id="querying-data" class="anchor" href="#querying-data" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Querying data</h2>
+<p>As a technique for query acceleration, Pre-aggregate tables cannot be queried directly.
+Queries are to be made on main table. While doing query planning, internally CarbonData will check
+associated pre-aggregate tables with the main table, and do query plan transformation accordingly.</p>
+<p>User can verify whether a query can leverage pre-aggregate table or not by executing <code>EXPLAIN</code>
+command, which will show the transformed logical plan, and thus user can check whether pre-aggregate
+table is selected.</p>
+<h2>
+<a id="compacting-pre-aggregate-tables" class="anchor" href="#compacting-pre-aggregate-tables" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Compacting pre-aggregate tables</h2>
+<p>Running Compaction command (<code>ALTER TABLE COMPACT</code>) on main table will <strong>not automatically</strong>
+compact the pre-aggregate tables created on the main table. User need to run Compaction command
+separately on each pre-aggregate table to compact them.</p>
+<p>Compaction is an optional operation for pre-aggregate table. If compaction is performed on
+main table but not performed on pre-aggregate table, all queries still can benefit from
+pre-aggregate tables. To further improve the query performance, compaction on pre-aggregate tables
+can be triggered to merge the segments and files in the pre-aggregate tables.</p>
+<h2>
+<a id="data-management-with-pre-aggregate-tables" class="anchor" href="#data-management-with-pre-aggregate-tables" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Data Management with pre-aggregate tables</h2>
+<p>In current implementation, data consistency needs to be maintained for both main table and pre-aggregate
+tables. Once there is pre-aggregate table created on the main table, following command on the main
+table
+is not supported:</p>
+<ol>
+<li>Data management command: <code>UPDATE/DELETE/DELETE SEGMENT</code>.</li>
+<li>Schema management command: <code>ALTER TABLE DROP COLUMN</code>, <code>ALTER TABLE CHANGE DATATYPE</code>,
+<code>ALTER TABLE RENAME</code>. Note that adding a new column is supported, and for dropping columns and
+change datatype command, CarbonData will check whether it will impact the pre-aggregate table, if
+not, the operation is allowed, otherwise operation will be rejected by throwing exception.</li>
+<li>Partition management command: <code>ALTER TABLE ADD/DROP PARTITION</code>
+</li>
+<li>Complex Datatypes for preaggregate is not supported.</li>
+</ol>
+<p>However, there is still way to support these operations on main table, in current CarbonData
+release, user can do as following:</p>
+<ol>
+<li>Remove the pre-aggregate table by <code>DROP DATAMAP</code> command</li>
+<li>Carry out the data management operation on main table</li>
+<li>Create the pre-aggregate table again by <code>CREATE DATAMAP</code> command
+Basically, user can manually trigger the operation by re-building the datamap.</li>
+</ol>
+<script>
+$(function() {
+ // Show selected style on nav item
+ $('.b-nav__datamap').addClass('selected');
+
+ if (!$('.b-nav__datamap').parent().hasClass('nav__item__with__subs--expanded')) {
+ // Display datamap subnav items
+ $('.b-nav__datamap').parent().toggleClass('nav__item__with__subs--expanded');
+ }
+});
+</script></div>
+</div>
+</div>
+</div>
+<div class="doc-footer">
+ <a href="#top" class="scroll-top">Top</a>
+</div>
+</div>
+</section>
+</div>
+</div>
+</div>
+</section><!-- End systemblock part -->
+<script src="js/custom.js"></script>
+</body>
+</html>
\ No newline at end of file
diff --git a/src/main/webapp/timeseries-datamap-guide.html b/src/main/webapp/timeseries-datamap-guide.html
new file mode 100644
index 0000000..a9137d0
--- /dev/null
+++ b/src/main/webapp/timeseries-datamap-guide.html
@@ -0,0 +1,369 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href='images/favicon.ico' rel='shortcut icon' type='image/x-icon'>
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>CarbonData</title>
+ <style>
+
+ </style>
+ <!-- Bootstrap -->
+
+ <link rel="stylesheet" href="css/bootstrap.min.css">
+ <link href="css/style.css" rel="stylesheet">
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.scom/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ <script src="js/jquery.min.js"></script>
+ <script src="js/bootstrap.min.js"></script>
+ <script defer src="https://use.fontawesome.com/releases/v5.0.8/js/all.js"></script>
+
+
+</head>
+<body>
+<header>
+ <nav class="navbar navbar-default navbar-custom cd-navbar-wrapper">
+ <div class="container">
+ <div class="navbar-header">
+ <button aria-controls="navbar" aria-expanded="false" data-target="#navbar" data-toggle="collapse"
+ class="navbar-toggle collapsed" type="button">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a href="index.html" class="logo">
+ <img src="images/CarbonDataLogo.png" alt="CarbonData logo" title="CarbocnData logo"/>
+ </a>
+ </div>
+ <div class="navbar-collapse collapse cd_navcontnt" id="navbar">
+ <ul class="nav navbar-nav navbar-right navlist-custom">
+ <li><a href="index.html" class="hidden-xs"><i class="fa fa-home" aria-hidden="true"></i> </a>
+ </li>
+ <li><a href="index.html" class="hidden-lg hidden-md hidden-sm">Home</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle " data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false"> Download <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.5.0/"
+ target="_blank">Apache CarbonData 1.5.0</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.1/"
+ target="_blank">Apache CarbonData 1.4.1</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.4.0/"
+ target="_blank">Apache CarbonData 1.4.0</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.1/"
+ target="_blank">Apache CarbonData 1.3.1</a></li>
+ <li>
+ <a href="https://dist.apache.org/repos/dist/release/carbondata/1.3.0/"
+ target="_blank">Apache CarbonData 1.3.0</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/Releases"
+ target="_blank">Release Archive</a></li>
+ </ul>
+ </li>
+ <li><a href="documentation.html" class="active">Documentation</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
+ aria-expanded="false">Community <span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li>
+ <a href="https://github.com/apache/carbondata/blob/master/docs/how-to-contribute-to-apache-carbondata.md"
+ target="_blank">Contributing to CarbonData</a></li>
+ <li>
+ <a href="https://github.com/apache/carbondata/blob/master/docs/release-guide.md"
+ target="_blank">Release Guide</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/display/CARBONDATA/PMC+and+Committers+member+list"
+ target="_blank">Project PMC and Committers</a></li>
+ <li>
+ <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=66850609"
+ target="_blank">CarbonData Meetups</a></li>
+ <li><a href="security.html">Apache CarbonData Security</a></li>
+ <li><a href="https://issues.apache.org/jira/browse/CARBONDATA" target="_blank">Apache
+ Jira</a></li>
+ <li><a href="videogallery.html">CarbonData Videos </a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="apache_link hidden-xs dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+
+ <li class="dropdown">
+ <a href="http://www.apache.org/" class="hidden-lg hidden-md hidden-sm dropdown-toggle"
+ data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <ul class="dropdown-menu">
+ <li><a href="http://www.apache.org/" target="_blank">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/" target="_blank">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html"
+ target="_blank">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
+ </ul>
+ </li>
+
+ <li>
+ <a href="#" id="search-icon"><i class="fa fa-search" aria-hidden="true"></i></a>
+
+ </li>
+
+ </ul>
+ </div><!--/.nav-collapse -->
+ <div id="search-box">
+ <form method="get" action="http://www.google.com/search" target="_blank">
+ <div class="search-block">
+ <table border="0" cellpadding="0" width="100%">
+ <tr>
+ <td style="width:80%">
+ <input type="text" name="q" size=" 5" maxlength="255" value=""
+ class="search-input" placeholder="Search...." required/>
+ </td>
+ <td style="width:20%">
+ <input type="submit" value="Search"/></td>
+ </tr>
+ <tr>
+ <td align="left" style="font-size:75%" colspan="2">
+ <input type="checkbox" name="sitesearch" value="carbondata.apache.org" checked/>
+ <span style=" position: relative; top: -3px;"> Only search for CarbonData</span>
+ </td>
+ </tr>
+ </table>
+ </div>
+ </form>
+ </div>
+ </div>
+ </nav>
+</header> <!-- end Header part -->
+
+<div class="fixed-padding"></div> <!-- top padding with fixde header -->
+
+<section><!-- Dashboard nav -->
+ <div class="container-fluid q">
+ <div class="col-sm-12 col-md-12 maindashboard">
+ <div class="verticalnavbar">
+ <nav class="b-sticky-nav">
+ <div class="nav-scroller">
+ <div class="nav__inner">
+ <a class="b-nav__intro nav__item" href="./introduction.html">introduction</a>
+ <a class="b-nav__quickstart nav__item" href="./quick-start-guide.html">quick start</a>
+ <a class="b-nav__uses nav__item" href="./usecases.html">use cases</a>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__docs nav__item nav__sub__anchor" href="./language-manual.html">Language Reference</a>
+ <a class="nav__item nav__sub__item" href="./ddl-of-carbondata.html">DDL</a>
+ <a class="nav__item nav__sub__item" href="./dml-of-carbondata.html">DML</a>
+ <a class="nav__item nav__sub__item" href="./streaming-guide.html">Streaming</a>
+ <a class="nav__item nav__sub__item" href="./configuration-parameters.html">Configuration</a>
+ <a class="nav__item nav__sub__item" href="./datamap-developer-guide.html">Datamaps</a>
+ <a class="nav__item nav__sub__item" href="./supported-data-types-in-carbondata.html">Data Types</a>
+ </div>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__datamap nav__item nav__sub__anchor" href="./datamap-management.html">DataMaps</a>
+ <a class="nav__item nav__sub__item" href="./bloomfilter-datamap-guide.html">Bloom Filter</a>
+ <a class="nav__item nav__sub__item" href="./lucene-datamap-guide.html">Lucene</a>
+ <a class="nav__item nav__sub__item" href="./preaggregate-datamap-guide.html">Pre-Aggregate</a>
+ <a class="nav__item nav__sub__item" href="./timeseries-datamap-guide.html">Time Series</a>
+ </div>
+
+ <div class="nav__item nav__item__with__subs">
+ <a class="b-nav__api nav__item nav__sub__anchor" href="./sdk-guide.html">API</a>
+ <a class="nav__item nav__sub__item" href="./sdk-guide.html">Java SDK</a>
+ <a class="nav__item nav__sub__item" href="./CSDK-guide.html">C++ SDK</a>
+ </div>
+
+ <a class="b-nav__perf nav__item" href="./performance-tuning.html">Performance Tuning</a>
+ <a class="b-nav__s3 nav__item" href="./s3-guide.html">S3 Storage</a>
+ <a class="b-nav__faq nav__item" href="./faq.html">FAQ</a>
+ <a class="b-nav__contri nav__item" href="./how-to-contribute-to-apache-carbondata.html">Contribute</a>
+ <a class="b-nav__security nav__item" href="./security.html">Security</a>
+ <a class="b-nav__release nav__item" href="./release-guide.html">Release Guide</a>
+ </div>
+ </div>
+ <div class="navindicator">
+ <div class="b-nav__intro navindicator__item"></div>
+ <div class="b-nav__quickstart navindicator__item"></div>
+ <div class="b-nav__uses navindicator__item"></div>
+ <div class="b-nav__docs navindicator__item"></div>
+ <div class="b-nav__datamap navindicator__item"></div>
+ <div class="b-nav__api navindicator__item"></div>
+ <div class="b-nav__perf navindicator__item"></div>
+ <div class="b-nav__s3 navindicator__item"></div>
+ <div class="b-nav__faq navindicator__item"></div>
+ <div class="b-nav__contri navindicator__item"></div>
+ <div class="b-nav__security navindicator__item"></div>
+ </div>
+ </nav>
+ </div>
+ <div class="mdcontent">
+ <section>
+ <div style="padding:10px 15px;">
+ <div id="viewpage" name="viewpage">
+ <div class="row">
+ <div class="col-sm-12 col-md-12">
+ <div>
+<h1>
+<a id="carbondata-timeseries-datamap" class="anchor" href="#carbondata-timeseries-datamap" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CarbonData Timeseries DataMap</h1>
+<ul>
+<li><a href="#timeseries-datamap-introduction-alpha-feature">Timeseries DataMap Introduction</a></li>
+<li><a href="#compacting-timeseries-datamp">Compaction</a></li>
+<li><a href="#data-management-on-timeseries-datamap">Data Management</a></li>
+</ul>
+<h2>
+<a id="timeseries-datamap-introduction-alpha-feature" class="anchor" href="#timeseries-datamap-introduction-alpha-feature" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Timeseries DataMap Introduction (Alpha Feature)</h2>
+<p>Timeseries DataMap is a pre-aggregate table implementation based on 'pre-aggregate' DataMap.
+Difference is that Timeseries DataMap has built-in understanding of time hierarchy and
+levels: year, month, day, hour, minute, so that it supports automatic roll-up in time dimension
+for query.</p>
+<p><strong>CAUTION:</strong> Current version of CarbonData does not support roll-up.It will be implemented in future versions.</p>
+<p>The data loading, querying, compaction command and its behavior is the same as preaggregate DataMap.
+Please refer to <a href="./preaggregate-datamap-guide.html">Pre-aggregate DataMap</a>
+for more information.</p>
+<p>To use this datamap, user can create multiple timeseries datamap on the main table which has
+a <em>event_time</em> column, one datamap for one time granularity.</p>
+<p>For example, below statement effectively create multiple pre-aggregate tables on main table called
+<strong>timeseries</strong></p>
+<pre><code>CREATE DATAMAP agg_year
+ON TABLE sales
+USING "timeseries"
+DMPROPERTIES (
+ 'event_time'='order_time',
+ 'year_granularity'='1',
+) AS
+SELECT order_time, country, sex, sum(quantity), max(quantity), count(user_id), sum(price),
+ avg(price) FROM sales GROUP BY order_time, country, sex
+
+CREATE DATAMAP agg_month
+ON TABLE sales
+USING "timeseries"
+DMPROPERTIES (
+ 'event_time'='order_time',
+ 'month_granularity'='1',
+) AS
+SELECT order_time, country, sex, sum(quantity), max(quantity), count(user_id), sum(price),
+ avg(price) FROM sales GROUP BY order_time, country, sex
+
+CREATE DATAMAP agg_day
+ON TABLE sales
+USING "timeseries"
+DMPROPERTIES (
+ 'event_time'='order_time',
+ 'day_granularity'='1',
+) AS
+SELECT order_time, country, sex, sum(quantity), max(quantity), count(user_id), sum(price),
+ avg(price) FROM sales GROUP BY order_time, country, sex
+
+CREATE DATAMAP agg_sales_hour
+ON TABLE sales
+USING "timeseries"
+DMPROPERTIES (
+ 'event_time'='order_time',
+ 'hour_granularity'='1',
+) AS
+SELECT order_time, country, sex, sum(quantity), max(quantity), count(user_id), sum(price),
+ avg(price) FROM sales GROUP BY order_time, country, sex
+
+CREATE DATAMAP agg_minute
+ON TABLE sales
+USING "timeseries"
+DMPROPERTIES (
+ 'event_time'='order_time',
+ 'minute_granularity'='1',
+) AS
+SELECT order_time, country, sex, sum(quantity), max(quantity), count(user_id), sum(price),
+ avg(price) FROM sales GROUP BY order_time, country, sex
+</code></pre>
+<p>For querying timeseries data, Carbondata has builtin support for following time related UDF</p>
+<pre><code>timeseries(timeseries column name, 'aggregation level')
+</code></pre>
+<pre><code>SELECT timeseries(order_time, 'hour'), sum(quantity) FROM sales GROUP BY timeseries(order_time,
+'hour')
+</code></pre>
+<p>It is <strong>not necessary</strong> to create pre-aggregate tables for each granularity unless required for
+query.</p>
+<p>For Example: For main table <strong>sales</strong> , if following timeseries datamaps were created for day
+level and hour level pre-aggregate</p>
+<pre><code> CREATE DATAMAP agg_day
+ ON TABLE sales
+ USING "timeseries"
+ DMPROPERTIES (
+ 'event_time'='order_time',
+ 'day_granularity'='1',
+ ) AS
+ SELECT order_time, country, sex, sum(quantity), max(quantity), count(user_id), sum(price),
+ avg(price) FROM sales GROUP BY order_time, country, sex
+
+ CREATE DATAMAP agg_sales_hour
+ ON TABLE sales
+ USING "timeseries"
+ DMPROPERTIES (
+ 'event_time'='order_time',
+ 'hour_granularity'='1',
+ ) AS
+ SELECT order_time, country, sex, sum(quantity), max(quantity), count(user_id), sum(price),
+ avg(price) FROM sales GROUP BY order_time, country, sex
+</code></pre>
+<p>Queries like below will not be rolled-up and hit the main table</p>
+<pre><code>Select timeseries(order_time, 'month'), sum(quantity) from sales group by timeseries(order_time,
+ 'month')
+
+Select timeseries(order_time, 'year'), sum(quantity) from sales group by timeseries(order_time,
+ 'year')
+</code></pre>
+<p>NOTE (<b>RESTRICTION</b>):</p>
+<ul>
+<li>Only value of 1 is supported for hierarchy levels. Other hierarchy levels will be supported in
+the future CarbonData release.</li>
+<li>timeseries datamap for the desired levels needs to be created one after the other</li>
+<li>timeseries datamaps created for each level needs to be dropped separately</li>
+</ul>
+<h2>
+<a id="compacting-timeseries-datamp" class="anchor" href="#compacting-timeseries-datamp" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Compacting timeseries datamp</h2>
+<p>Refer to Compaction section in <a href="./preaggregate-datamap-guide.html">preaggregation datamap</a>.
+Same applies to timeseries datamap.</p>
+<h2>
+<a id="data-management-on-timeseries-datamap" class="anchor" href="#data-management-on-timeseries-datamap" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Data Management on timeseries datamap</h2>
+<p>Refer to Data Management section in <a href="./preaggregate-datamap-guide.html">preaggregation datamap</a>.
+Same applies to timeseries datamap.</p>
+<script>
+$(function() {
+ // Show selected style on nav item
+ $('.b-nav__datamap').addClass('selected');
+
+ if (!$('.b-nav__datamap').parent().hasClass('nav__item__with__subs--expanded')) {
+ // Display datamap subnav items
+ $('.b-nav__datamap').parent().toggleClass('nav__item__with__subs--expanded');
+ }
+});
+</script></div>
+</div>
+</div>
+</div>
+<div class="doc-footer">
+ <a href="#top" class="scroll-top">Top</a>
+</div>
+</div>
+</section>
+</div>
+</div>
+</div>
+</section><!-- End systemblock part -->
+<script src="js/custom.js"></script>
+</body>
+</html>
\ No newline at end of file