You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kylin.apache.org by li...@apache.org on 2016/04/08 09:10:12 UTC
svn commit: r1738214 - in /kylin/site: docs15/install/advance_settings.html
feed.xml
Author: lidong
Date: Fri Apr 8 07:10:12 2016
New Revision: 1738214
URL: http://svn.apache.org/viewvc?rev=1738214&view=rev
Log:
revise advanced settings
Modified:
kylin/site/docs15/install/advance_settings.html
kylin/site/feed.xml
Modified: kylin/site/docs15/install/advance_settings.html
URL: http://svn.apache.org/viewvc/kylin/site/docs15/install/advance_settings.html?rev=1738214&r1=1738213&r2=1738214&view=diff
==============================================================================
--- kylin/site/docs15/install/advance_settings.html (original)
+++ kylin/site/docs15/install/advance_settings.html Fri Apr 8 07:10:12 2016
@@ -1864,9 +1864,17 @@
<article class="post-content" >
- <h2 id="enable-lzo-compression">Enable LZO compression</h2>
+ <h2 id="settings-for-compression">Settings for compression</h2>
-<p>By default Kylin leverages snappy compression to compress the output of MR jobs, as well as hbase table storage, reducing the storage overhead. We do not choose LZO compression in Kylin because hadoop venders tend to not include LZO in their distributions due to license(GPL) issues. To enable LZO in Kylin, follow these steps:</p>
+<h3 id="disable-compression-easiest-way-to-address-compression-related-issues">Disable compression (Easiest way to address compression related issues)</h3>
+
+<p>By default Kylin leverages snappy compression to compress the output of MR jobs, as well as hbase table storage, to reduce the storage overhead. We do not choose LZO compression in Kylin because hadoop venders tend to not include LZO in their distributions due to license(GPL) issues. If you compression related issues happened in your cubing job, you have two options: 1. Disable compression 2. Choose other compression algorithms like LZO.</p>
+
+<h4 id="disable-hbase-compression">Disable HBase compression</h4>
+
+<p>Compression settings only take effect after restarting Kylin server instance (by <code class="highlighter-rouge">./kylin.sh start</code> and <code class="highlighter-rouge">./kylin.sh stop</code>). To disable compressing MR jobs you need to modify $KYLIN_HOME/conf/kylin_job_conf.xml by removing all configuration entries related to compression(Just grep the keyword âcompressâ). To disable compressing hbase tables you need to open $KYLIN_HOME/conf/kylin.properties and remove the line starting with kylin.hbase.default.compression.codec.</p>
+
+<h3 id="lzo-compression">LZO compression</h3>
<h4 id="make-sure-lzo-is-working-in-your-environment">Make sure LZO is working in your environment</h4>
@@ -1880,18 +1888,9 @@ To test if the hbase cluster is ready to
<div class="highlight"><pre><code class="language-groff" data-lang="groff">create 'lzoTable', {NAME => 'colFam',COMPRESSION => 'LZO'}</code></pre></div>
-<h4 id="use-lzo-for-hbase-compression">Use LZO for HBase compression</h4>
-
-<p>Youâll need to stop Kylin first by running <code class="highlighter-rouge">./kylin.sh stop</code>, and then modify $KYLIN_HOME/conf/kylin_job_conf.xml by uncommenting some configuration entries related to LZO compression. <br />
-After this, you need to run <code class="highlighter-rouge">./kylin.sh start</code> to start Kylin again. Now Kylin will use LZO to compress MR outputs and hbase tables.</p>
-
-<p>Goto $KYLIN_HOME/conf/kylin.properties, change kylin.hbase.default.compression.codec=snappy to kylin.hbase.default.compression.codec=lzo</p>
-
-<h4 id="use-lzo-for-mr-jobs">Use LZO for MR jobs</h4>
-
-<p>Modify $KYLIN_HOME/conf/kylin_job_conf.xml by changing all org.apache.hadoop.io.compress.SnappyCodec to com.hadoop.compression.lzo.LzoCodec.</p>
+<h4 id="use-lzo-compression">Use LZO compression</h4>
-<p>Start Kylin again. Now Kylin will use LZO to compress MR outputs and HBase tables.</p>
+<p>Compression settings only take effect after restarting Kylin server instance (by <code class="highlighter-rouge">./kylin.sh start</code> and <code class="highlighter-rouge">./kylin.sh stop</code>). To use LZO for compressing MR jobs you need to modify $KYLIN_HOME/conf/kylin_job_conf.xml by replacing configuration entries related to compression from org.apache.hadoop.io.compress.SnappyCodec to com.hadoop.compression.lzo. You can refer to other documents for details: http://xiaming.me/posts/2014/05/03/enable-lzo-compression-on-hadoop-pig-and-spark. To use LZO for compressing hbase tables you need to open $KYLIN_HOME/conf/kylin.properties, change kylin.hbase.default.compression.codec=snappy to kylin.hbase.default.compression.codec=lzo.</p>
<h2 id="enable-ldap-or-sso-authentication">Enable LDAP or SSO authentication</h2>
Modified: kylin/site/feed.xml
URL: http://svn.apache.org/viewvc/kylin/site/feed.xml?rev=1738214&r1=1738213&r2=1738214&view=diff
==============================================================================
--- kylin/site/feed.xml (original)
+++ kylin/site/feed.xml Fri Apr 8 07:10:12 2016
@@ -19,8 +19,8 @@
<description>Apache Kylin Home</description>
<link>http://kylin.apache.org/</link>
<atom:link href="http://kylin.apache.org/feed.xml" rel="self" type="application/rss+xml"/>
- <pubDate>Fri, 08 Apr 2016 06:59:44 -0700</pubDate>
- <lastBuildDate>Fri, 08 Apr 2016 06:59:44 -0700</lastBuildDate>
+ <pubDate>Fri, 08 Apr 2016 08:08:46 -0700</pubDate>
+ <lastBuildDate>Fri, 08 Apr 2016 08:08:46 -0700</lastBuildDate>
<generator>Jekyll v2.5.3</generator>
<item>