You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kylin.apache.org by sh...@apache.org on 2017/01/05 02:24:08 UTC
kylin git commit: update advanced setting page for hive config
overwirte
Repository: kylin
Updated Branches:
refs/heads/document 7e234260a -> 6ece3105f
update advanced setting page for hive config overwirte
Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/6ece3105
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/6ece3105
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/6ece3105
Branch: refs/heads/document
Commit: 6ece3105f952f006fc9338fb55263d4763f7d7fa
Parents: 7e23426
Author: shaofengshi <sh...@apache.org>
Authored: Thu Jan 5 10:24:03 2017 +0800
Committer: shaofengshi <sh...@apache.org>
Committed: Thu Jan 5 10:24:03 2017 +0800
----------------------------------------------------------------------
website/_docs15/install/advance_settings.md | 5 +++++
website/_docs16/install/advance_settings.md | 6 ++++++
2 files changed, 11 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/kylin/blob/6ece3105/website/_docs15/install/advance_settings.md
----------------------------------------------------------------------
diff --git a/website/_docs15/install/advance_settings.md b/website/_docs15/install/advance_settings.md
index 2b9b383..0c6a9d6 100644
--- a/website/_docs15/install/advance_settings.md
+++ b/website/_docs15/install/advance_settings.md
@@ -22,6 +22,11 @@ The `conf/kylin_job_conf.xml` and `conf/kylin_job_conf_inmem.xml` manage the def
* If want a cube's job getting more memory from Yarn, you can define: `kylin.job.mr.config.override.mapreduce.map.java.opts=-Xmx7g` and `kylin.job.mr.config.override.mapreduce.map.memory.mb=8192`
* If want a cube's job going to a different Yarn resource queue, you can define: `kylin.job.mr.config.override.mapreduce.job.queuename=myQueue` (note: "myQueue" is just a sample)
+ ## Overwrite default Hive job conf at Cube level
+The `conf/kylin_hive_conf.xml` manage the default configurations when running Hive job (like creating intermediate flat hive table). If you have the need to customize the configs by cube, you can achieve that with the similar way as above, but need using another prefix `kylin.hive.config.override.`; These configs will be parsed out and then applied when running "hive -e" or "beeline" commands. See example below:
+
+ * If want hive goes a different Yarn resource queue, you can define: `kylin.hive.config.override.mapreduce.job.queuename=myQueue` (note: "myQueue" is just a sample)
+
## Enable compression
By default, Kylin does not enable compression, this is not the recommend settings for production environment, but a tradeoff for new Kylin users. A suitable compression algorithm will reduce the storage overhead. But unsupported algorithm will break the Kylin job build also. There are three kinds of compression used in Kylin, HBase table compression, Hive output compression and MR jobs output compression.
http://git-wip-us.apache.org/repos/asf/kylin/blob/6ece3105/website/_docs16/install/advance_settings.md
----------------------------------------------------------------------
diff --git a/website/_docs16/install/advance_settings.md b/website/_docs16/install/advance_settings.md
index 306637d..8699f75 100644
--- a/website/_docs16/install/advance_settings.md
+++ b/website/_docs16/install/advance_settings.md
@@ -22,6 +22,12 @@ The `conf/kylin_job_conf.xml` and `conf/kylin_job_conf_inmem.xml` manage the def
* If want a cube's job getting more memory from Yarn, you can define: `kylin.job.mr.config.override.mapreduce.map.java.opts=-Xmx7g` and `kylin.job.mr.config.override.mapreduce.map.memory.mb=8192`
* If want a cube's job going to a different Yarn resource queue, you can define: `kylin.job.mr.config.override.mapreduce.job.queuename=myQueue` (note: "myQueue" is just a sample)
+ ## Overwrite default Hive job conf at Cube level
+The `conf/kylin_hive_conf.xml` manage the default configurations when running Hive job (like creating intermediate flat hive table). If you have the need to customize the configs by cube, you can achieve that with the similar way as above, but need using another prefix `kylin.hive.config.override.`; These configs will be parsed out and then applied when running "hive -e" or "beeline" commands. See example below:
+
+ * If want hive goes a different Yarn resource queue, you can define: `kylin.hive.config.override.mapreduce.job.queuename=myQueue` (note: "myQueue" is just a sample)
+
+
## Enable compression
By default, Kylin does not enable compression, this is not the recommend settings for production environment, but a tradeoff for new Kylin users. A suitable compression algorithm will reduce the storage overhead. But unsupported algorithm will break the Kylin job build also. There are three kinds of compression used in Kylin, HBase table compression, Hive output compression and MR jobs output compression.