You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kylin.apache.org by li...@apache.org on 2015/05/12 04:00:07 UTC
[1/2] incubator-kylin git commit: KYLIN-757 Cache wasn't flushed in
cluster mode
Repository: incubator-kylin
Updated Branches:
refs/heads/0.7.1 8744f6018 -> 124121764
KYLIN-757 Cache wasn't flushed in cluster mode
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/7ee27ad6
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/7ee27ad6
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/7ee27ad6
Branch: refs/heads/0.7.1
Commit: 7ee27ad6054bbf966d791d2303ede4d58283aeae
Parents: 8744f60
Author: Shao Feng, Shi <sh...@ebay.com>
Authored: Fri May 8 18:10:29 2015 +0800
Committer: Shao Feng, Shi <sh...@ebay.com>
Committed: Fri May 8 18:10:29 2015 +0800
----------------------------------------------------------------------
common/src/main/java/org/apache/kylin/common/KylinConfig.java | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/7ee27ad6/common/src/main/java/org/apache/kylin/common/KylinConfig.java
----------------------------------------------------------------------
diff --git a/common/src/main/java/org/apache/kylin/common/KylinConfig.java b/common/src/main/java/org/apache/kylin/common/KylinConfig.java
index 113afd4..3e3f4a3 100644
--- a/common/src/main/java/org/apache/kylin/common/KylinConfig.java
+++ b/common/src/main/java/org/apache/kylin/common/KylinConfig.java
@@ -490,6 +490,7 @@ public class KylinConfig {
void reloadKylinConfig(InputStream is) {
PropertiesConfiguration config = new PropertiesConfiguration();
+ config.setListDelimiter((char)0); // disable list delimiter, Kylin will parse and split
try {
config.load(is);
} catch (ConfigurationException e) {
[2/2] incubator-kylin git commit: update document for "Kylin Metadata
Store"
Posted by li...@apache.org.
update document for "Kylin Metadata Store"
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/12412176
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/12412176
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/12412176
Branch: refs/heads/0.7.1
Commit: 124121764a3eb0652032c8add97f02708aa5fd3a
Parents: 7ee27ad
Author: Li, Yang <ya...@ebay.com>
Authored: Tue May 12 09:55:12 2015 +0800
Committer: Li, Yang <ya...@ebay.com>
Committed: Tue May 12 09:55:12 2015 +0800
----------------------------------------------------------------------
docs/Operations/Kylin Metadata Store.md | 28 ++++++++++++----------------
1 file changed, 12 insertions(+), 16 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/12412176/docs/Operations/Kylin Metadata Store.md
----------------------------------------------------------------------
diff --git a/docs/Operations/Kylin Metadata Store.md b/docs/Operations/Kylin Metadata Store.md
index bba3b58..79ec091 100644
--- a/docs/Operations/Kylin Metadata Store.md
+++ b/docs/Operations/Kylin Metadata Store.md
@@ -1,31 +1,27 @@
-Kylin organizes all of its metadata(including cube descriptions and instances, projects, inverted index description and instances, jobs, tables and dictionaries) as a hierarchy file system. However, Kylin uses hbase to store it, rather than normal file system. If you check your kylin configuration file(kylin.properties) you will find such a line:
+Kylin organizes all of its metadata (including cube descriptions and instances, projects, inverted index description and instances, jobs, tables and dictionaries) as a hierarchy of files, that are stored in HBase. You can backup and restore these metadata by download to local file system and upload again.
-`# The metadata store in hbase`
-`kylin.metadata.url=kylin_metadata@hbase:sandbox.hortonworks.com:2181:/hbase-unsecure`
+Check the `conf/kylin.properties`
-This indicates that the metadata will be saved as a htable called `kylin_metadata`. You can scan the htable in hbase shell to check it out.
+ kylin.metadata.url=kylin_metadata@hbase
-# Backup Metadata Store
+This indicates that the metadata will be saved as a HTable called `kylin_metadata`. You can scan the HTable in HBase shell.
-Sometimes you need to backup the Kylin's Metadata Store from hbase to your disk file system.
-In such cases, assuming you're on the hadoop CLI(or sandbox) where you deployed Kylin, you can use:
+# Backup Metadata Store
- mkdir ~/meta_dump
+Sometimes you want to backup Kylin's metadata. Below command downloads all metadata to local directory `~/meta_dump`.
- hbase org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-x.x.x-SNAPSHOT-job.jar org.apache.kylin.common.persistence.ResourceTool copy ${KYLIN_HOME}/conf/kylin.properties ~/meta_dump
+ hbase org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-x.x.x-SNAPSHOT-job.jar org.apache.kylin.common.persistence.ResourceTool download ~/meta_dump
-to dump your metadata to your local folder ~/meta_dump.
+Add `-Dexclude=/dict,/job_output,/table_snapshot` flag to the command to exclude certain metadata sub-directories.
# Restore Metadata Store
-In case you find your metadata store messed up, and you want to restore to a previous backup:
-
-first clean up the metadata store:
+To restore a backup, first clean up the metadata store.
- hbase org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-x.x.x-SNAPSHOT-job.jar org.apache.kylin.common.persistence.ResourceTool reset
+ hbase org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-x.x.x-SNAPSHOT-job.jar org.apache.kylin.common.persistence.ResourceTool reset
-then upload the backup metadata in ~/meta_dump to Kylin's metadata store:
+Then upload the backup metadata from local file system.
- hbase org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-x.x.x-SNAPSHOT-job.jar org.apache.kylin.common.persistence.ResourceTool copy ~/meta_dump ${KYLIN_HOME}/conf/kylin.properties
+ hbase org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-x.x.x-SNAPSHOT-job.jar org.apache.kylin.common.persistence.ResourceTool upload ~/meta_dump