You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kylin.apache.org by li...@apache.org on 2019/08/08 14:10:46 UTC
svn commit: r1864708 - in /kylin/site: ./ docs30/howto/
images/Hive-Global-Dictionary/
Author: lidong
Date: Thu Aug 8 14:10:46 2019
New Revision: 1864708
URL: http://svn.apache.org/viewvc?rev=1864708&view=rev
Log:
Add detailed analysis for Hive Global Dictionary
Added:
kylin/site/images/Hive-Global-Dictionary/
kylin/site/images/Hive-Global-Dictionary/cube-level-config.png (with props)
kylin/site/images/Hive-Global-Dictionary/hive-global-dict-table.png (with props)
kylin/site/images/Hive-Global-Dictionary/set-hive-dict-column.png (with props)
kylin/site/images/Hive-Global-Dictionary/three-added-steps.png (with props)
Modified:
kylin/site/docs30/howto/howto_use_hive_mr_dict.html
kylin/site/feed.xml
Modified: kylin/site/docs30/howto/howto_use_hive_mr_dict.html
URL: http://svn.apache.org/viewvc/kylin/site/docs30/howto/howto_use_hive_mr_dict.html?rev=1864708&r1=1864707&r2=1864708&view=diff
==============================================================================
--- kylin/site/docs30/howto/howto_use_hive_mr_dict.html (original)
+++ kylin/site/docs30/howto/howto_use_hive_mr_dict.html Thu Aug 8 14:10:46 2019
@@ -6357,31 +6357,219 @@ var _hmt = _hmt || [];
<article class="post-content" >
- <h3 id="global-dictionary">Global Dictionary</h3>
-<p>Count distinct measure is very important for many scenario, such as PageView statistics, Kylin support count distinct since 1.5.3 (http://kylin.apache.org/blog/2016/08/01/count-distinct-in-kylin/). <br />
-Apache Kylin implements precisely count distinct based on bitmap, and use global dictionary to encode string value into a Dict. <br />
-Currently we have to build global dictionary in single process/JVM, which may take a lot of time and memory for UHC. By in this feature(KYLIN-3841), we use Hive, a distributed SQL engine to build global dictionary.</p>
+ <h2 id="global-dictionary-in-hive">Global Dictionary in Hive</h2>
+<p>Count distinct(bitmap) measure is very important for many scenario, such as PageView statistics, and Kylin support count distinct since 1.5.3 .<br />
+Apache Kylin implements precisely count distinct measure based on bitmap, and use global dictionary to encode string value into integer. <br />
+Currently we have to build global dictionary in single process/JVM, which may take a lot of time and memory for UHC. By this feature(KYLIN-3841), we use Hive, a distributed SQL engine to build global dictionary.</p>
<p>This will help to:<br />
-1. Reduce memory pressure of Kylin process, MapReduce will be used to build dict for Kylin<br />
-2. Make global dictionary reusable<br />
-3. Make global dictionary readable, you may use global dictionary outside Kylin, maybe useful in many scenario.</p>
-
-<p>And this feature will add three steps if enabled.<br />
-1. Global Dict Mr/Hive extract dict_val from Data<br />
-2. Global Dict Mr/Hive build dict_val<br />
-3. Global Dict Mr/Hive replace dict_val to Data</p>
-
-<h3 id="how-to-use">How to use</h3>
-<p>If you have a count distinct(bitmap) measure for a UHC. Says columns name are PV_ID and USER_ID, and table name is USER_ACTION, you may add cube-level configuration <code class="highlighter-rouge">kylin.dictionary.mr-hive.columns=USER_ACTION_PV_ID,USER_ACTION_USER_ID</code> to enable this feature.<br />
-You have to know that the value will be replaced into encoded integer in flat hive table, and this may cause failure of some query.</p>
+1. Reduce memory pressure of Kylin process, MapReduce(or other engine which hive used) will be used to build dict instead of Kylin process itself.<br />
+2. Make build base cuboid quicker, because string value has been encoded in previous step.<br />
+3. Make global dictionary reusable.<br />
+4. Make global dictionary readable and bijective, you may use global dictionary outside Kylin, this maybe useful in many scenario.</p>
+
+<h3 id="step-by-step-analysis">Step by step Analysis</h3>
+<p>This feature will add three additional steps in cube building if enabled, let us try to understand what Kylin do in these steps.</p>
+
+<ol>
+ <li>
+ <p>Global Dict Mr/Hive extract dict_val from Data</p>
+
+ <ul>
+ <li>Create a Hive table for store global dictionary if it is not exists, table name should be <em>CubeName_Suffix</em>. This table has two normal column and one partition column, two normal columns are <code class="highlighter-rouge">dict_key</code> and <code class="highlighter-rouge">dict_value</code>, which for origin value and encoded integer respectively.</li>
+ <li>Create a temporary table with â__group_byâ as its suffix, which used to store distinct value for specific column. This table has one normal column and one partition column, normal column is <code class="highlighter-rouge">dict_key</code> which used to store origin value.</li>
+ <li>Insert distinct value into temporary table created above for each column by using a hive query âselect cloA from flatTable group by cloAâ.</li>
+ </ul>
+
+ <p>When this step finished, you should get a temporary table contains distinct values, each partition for specific Count_Distinct column.</p>
+ </li>
+ <li>
+ <p>Global Dict Mr/Hive build dict_val</p>
+
+ <ul>
+ <li>Find all fresh distinct value which never exists in any older segments by <em>LEFT JOIN</em> between global dictionary table and temporary table.</li>
+ <li>Append all fresh distinct value to the tail of global dictionary table by <em>UNION</em>. By the power of <code class="highlighter-rouge">row_number</code> function in Hive, added value will be encoded with integer in incremental way.</li>
+ </ul>
+
+ <p>When this step finished, all distinct value for all Count_Distinct column will be encoded correctly in global dictionary table.</p>
+ </li>
+ <li>
+ <p>Global Dict Mr/Hive replace dict_val to Data</p>
+
+ <ul>
+ <li>Using <em>LEFT JOIN</em> to replace original string value with encoded integer on flat table which used to build cuboid later.</li>
+ </ul>
+
+ <p>When this step finished, all string value which belong to Count_Distinct column will be updated with encoded integer in flat hive table.</p>
+ </li>
+</ol>
+
+<hr />
+
+<h2 id="how-to-use">How to use</h2>
+
+<p>If you have some count distinct(bitmap) measure, and data type of that column is String, you may need Hive Global Dictionary. Says columns name are PV_ID and USER_ID, and table name is USER_ACTION, you may add cube-level configuration <code class="highlighter-rouge">kylin.dictionary.mr-hive.columns=USER_ACTION_PV_ID,USER_ACTION_USER_ID</code> to enable this feature.</p>
+
+<p>Please donât use hive global dictionary on integer type column, you have to know that the value will be replaced with encoded integer in flat hive table. If you have sum/max/min measure on the same column, you will get wrong result in these measures.</p>
+
+<p>And you should know this feature is conflicted with shrunken global dictionary(KYLIN-3491) because they fix the same thing in different way.</p>
<h3 id="configuration">Configuration</h3>
+
+<ul>
+ <li><code class="highlighter-rouge">kylin.dictionary.mr-hive.columns</code> is used to specific which columns need to use Hive-MR dict, should be <em>TABLE1_COLUMN1,TABLE2_COLUMN2</em>. Better configured in cube level, default value is empty.</li>
+ <li><code class="highlighter-rouge">kylin.dictionary.mr-hive.database</code> is used to specific which database Hive-MR dict table located, default value is <em>default</em>.</li>
+ <li><code class="highlighter-rouge">kylin.hive.union.style</code> Sometime sql which used to build global dict table may have problem in union syntax, you may refer to Hive Doc for more detail. The default value is <em>UNION</em>, using lower version of Hive should change to <em>UNION ALL</em>.</li>
+ <li><code class="highlighter-rouge">kylin.dictionary.mr-hive.table.suffix</code> is used to specific suffix of global dict table, default value is <em>_global_dict</em>.</li>
+</ul>
+
+<hr />
+
+<h2 id="screenshot">Screenshot</h2>
+
+<h4 id="sql-in-new-added-step-add-countdistinctbitmap-measure">SQL in new added step Add count_distinct(bitmap) measure</h4>
+
+<p><img src="/images/Hive-Global-Dictionary/cube-level-config.png" alt="add_count_distinct_bitmap" /></p>
+
+<h4 id="sql-in-new-added-step-set-hive-dict-column-in-cube-level-config">SQL in new added step Set hive-dict-column in cube level config</h4>
+
+<p><img src="/images/Hive-Global-Dictionary/set-hive-dict-column.png" alt="set-hive-dict-column" /></p>
+
+<h4 id="sql-in-new-added-step-three-added-steps-of-cubing-job">SQL in new added step Three added steps of cubing job</h4>
+
+<p><img src="/images/Hive-Global-Dictionary/three-added-steps.png" alt="three-added-steps" /></p>
+
+<h4 id="sql-in-new-added-step-hive-global-dictionary-table">SQL in new added step Hive Global Dictionary Table</h4>
+
+<p><img src="/images/Hive-Global-Dictionary/hive-global-dict-table.png" alt="hive-global-dict-table" /></p>
+
+<h4 id="sql-in-new-added-step">SQL in new added step</h4>
+
+<ul>
+ <li>Global Dict Mr/Hive extract dict_val from Data</li>
+</ul>
+
+<div class="highlight"><pre><code class="language-groff" data-lang="groff">CREATE TABLE IF NOT EXISTS lacus.KYLIN_SALE_HIVE_DICT_HIVE_GLOBAL
+ ( dict_key STRING COMMENT '',
+ dict_val INT COMMENT ''
+ )
+ COMMENT ''
+ PARTITIONED BY (dict_column string)
+ STORED AS TEXTFILE;
+ DROP TABLE IF EXISTS kylin_intermediate_kylin_sale_hive_dict_921b0a15_d7cd_a2e6_6852_4ce44158f195__group_by;
+ CREATE TABLE IF NOT EXISTS kylin_intermediate_kylin_sale_hive_dict_921b0a15_d7cd_a2e6_6852_4ce44158f195__group_by
+ (
+ dict_key STRING COMMENT ''
+ )
+ COMMENT ''
+ PARTITIONED BY (dict_column string)
+ STORED AS SEQUENCEFILE
+ ;
+ INSERT OVERWRITE TABLE kylin_intermediate_kylin_sale_hive_dict_921b0a15_d7cd_a2e6_6852_4ce44158f195__group_by
+ PARTITION (dict_column = 'KYLIN_SALES_LSTG_FORMAT_NAME')
+ SELECT
+ KYLIN_SALES_LSTG_FORMAT_NAME
+ FROM kylin_intermediate_kylin_sale_hive_dict_921b0a15_d7cd_a2e6_6852_4ce44158f195
+ GROUP BY KYLIN_SALES_LSTG_FORMAT_NAME
+ ;
+ INSERT OVERWRITE TABLE kylin_intermediate_kylin_sale_hive_dict_921b0a15_d7cd_a2e6_6852_4ce44158f195__group_by
+ PARTITION (dict_column = 'KYLIN_SALES_OPS_REGION')
+ SELECT
+ KYLIN_SALES_OPS_REGION
+ FROM kylin_intermediate_kylin_sale_hive_dict_921b0a15_d7cd_a2e6_6852_4ce44158f195
+ GROUP BY KYLIN_SALES_OPS_REGION ;</code></pre></div>
+
+<ul>
+ <li>Global Dict Mr/Hive build dict_val</li>
+</ul>
+
+<div class="highlight"><pre><code class="language-groff" data-lang="groff">INSERT OVERWRITE TABLE lacus.KYLIN_SALE_HIVE_DICT_HIVE_GLOBAL
+ PARTITION (dict_column = 'KYLIN_SALES_OPS_REGION')
+ SELECT dict_key, dict_val FROM lacus.KYLIN_SALE_HIVE_DICT_HIVE_GLOBAL
+ WHERE dict_column = 'KYLIN_SALES_OPS_REGION'
+ UNION ALL
+ SELECT a.dict_key as dict_key, (row_number() over(order by a.dict_key asc)) + (0) as dict_val
+ FROM
+ (
+ SELECT dict_key FROM default.kylin_intermediate_kylin_sale_hive_dict_921b0a15_d7cd_a2e6_6852_4ce44158f195__group_by WHERE dict_column = 'KYLIN_SALES_OPS_REGION' AND dict_key is not null
+ ) a
+ LEFT JOIN
+ (
+ SELECT dict_key, dict_val FROM lacus.KYLIN_SALE_HIVE_DICT_HIVE_GLOBAL WHERE dict_column = 'KYLIN_SALES_OPS_REGION'
+ ) b
+ ON a.dict_key = b.dict_key
+ WHERE b.dict_val is null;
+
+ INSERT OVERWRITE TABLE lacus.KYLIN_SALE_HIVE_DICT_HIVE_GLOBAL
+ PARTITION (dict_column = 'KYLIN_SALES_LSTG_FORMAT_NAME')
+ SELECT dict_key, dict_val FROM lacus.KYLIN_SALE_HIVE_DICT_HIVE_GLOBAL
+ WHERE dict_column = 'KYLIN_SALES_LSTG_FORMAT_NAME'
+ UNION ALL
+ SELECT a.dict_key as dict_key, (row_number() over(order by a.dict_key asc)) + (0) as dict_val
+ FROM
+ (
+ SELECT dict_key FROM default.kylin_intermediate_kylin_sale_hive_dict_921b0a15_d7cd_a2e6_6852_4ce44158f195__group_by WHERE dict_column = 'KYLIN_SALES_LSTG_FORMAT_NAME' AND dict_key is not null
+ ) a
+ LEFT JOIN
+ (
+ SELECT dict_key, dict_val FROM lacus.KYLIN_SALE_HIVE_DICT_HIVE_GLOBAL WHERE dict_column = 'KYLIN_SALES_LSTG_FORMAT_NAME'
+ ) b
+ ON a.dict_key = b.dict_key
+ WHERE b.dict_val is null;</code></pre></div>
+
+<ul>
+ <li>Global Dict Mr/Hive replace dict_val to Data</li>
+</ul>
+
+<div class="highlight"><pre><code class="language-groff" data-lang="groff">INSERT OVERWRITE TABLE default.kylin_intermediate_kylin_sale_hive_dict_921b0a15_d7cd_a2e6_6852_4ce44158f195
+ SELECT
+ a.KYLIN_SALES_TRANS_ID
+ ,a.KYLIN_SALES_PART_DT
+ ,a.KYLIN_SALES_LEAF_CATEG_ID
+ ,a.KYLIN_SALES_LSTG_SITE_ID
+ ,a.KYLIN_SALES_SELLER_ID
+ ,a.KYLIN_SALES_BUYER_ID
+ ,a.BUYER_ACCOUNT_ACCOUNT_COUNTRY
+ ,a.SELLER_ACCOUNT_ACCOUNT_COUNTRY
+ ,a.KYLIN_SALES_PRICE
+ ,a.KYLIN_SALES_ITEM_COUNT
+ ,a.KYLIN_SALES_LSTG_FORMAT_NAME
+ ,b. dict_val
+ FROM default.kylin_intermediate_kylin_sale_hive_dict_921b0a15_d7cd_a2e6_6852_4ce44158f195 a
+ LEFT OUTER JOIN
+ (
+ SELECT dict_key, dict_val FROM lacus.KYLIN_SALE_HIVE_DICT_HIVE_GLOBAL WHERE dict_column = 'KYLIN_SALES_OPS_REGION'
+ ) b
+ ON a.KYLIN_SALES_OPS_REGION = b.dict_key;
+ INSERT OVERWRITE TABLE default.kylin_intermediate_kylin_sale_hive_dict_921b0a15_d7cd_a2e6_6852_4ce44158f195
+ SELECT
+ a.KYLIN_SALES_TRANS_ID
+ ,a.KYLIN_SALES_PART_DT
+ ,a.KYLIN_SALES_LEAF_CATEG_ID
+ ,a.KYLIN_SALES_LSTG_SITE_ID
+ ,a.KYLIN_SALES_SELLER_ID
+ ,a.KYLIN_SALES_BUYER_ID
+ ,a.BUYER_ACCOUNT_ACCOUNT_COUNTRY
+ ,a.SELLER_ACCOUNT_ACCOUNT_COUNTRY
+ ,a.KYLIN_SALES_PRICE
+ ,a.KYLIN_SALES_ITEM_COUNT
+ ,b. dict_val
+ ,a.KYLIN_SALES_OPS_REGION
+ FROM default.kylin_intermediate_kylin_sale_hive_dict_921b0a15_d7cd_a2e6_6852_4ce44158f195 a
+ LEFT OUTER JOIN
+ (
+ SELECT dict_key, dict_val FROM lacus.KYLIN_SALE_HIVE_DICT_HIVE_GLOBAL WHERE dict_column = 'KYLIN_SALES_LSTG_FORMAT_NAME'
+ ) b
+ ON a.KYLIN_SALES_LSTG_FORMAT_NAME = b.dict_key;</code></pre></div>
+
+<h3 id="reference-link">Reference Link</h3>
+
<ul>
- <li><code class="highlighter-rouge">kylin.dictionary.mr-hive.columns</code> is used to specific which columns need to be Hive-MR dict.</li>
- <li><code class="highlighter-rouge">kylin.dictionary.mr-hive.database</code> is used to specific which database Hive-MR dict located.</li>
- <li><code class="highlighter-rouge">kylin.hive.union.style</code> Sometime sql which used to build global dict table may have syntax problem. This should be fixed by specific this entry with <em>UNION ALL</em>.</li>
- <li><code class="highlighter-rouge">kylin.dictionary.mr-hive.table.suffix</code> is used to specific suffix of global dict table.</li>
+ <li>https://issues.apache.org/jira/browse/KYLIN-3491</li>
+ <li>https://issues.apache.org/jira/browse/KYLIN-3841</li>
+ <li>https://issues.apache.org/jira/browse/KYLIN-3905</li>
+ <li>https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Union</li>
+ <li>http://kylin.apache.org/blog/2016/08/01/count-distinct-in-kylin/</li>
</ul>
</article>
Modified: kylin/site/feed.xml
URL: http://svn.apache.org/viewvc/kylin/site/feed.xml?rev=1864708&r1=1864707&r2=1864708&view=diff
==============================================================================
--- kylin/site/feed.xml (original)
+++ kylin/site/feed.xml Thu Aug 8 14:10:46 2019
@@ -19,8 +19,8 @@
<description>Apache Kylin Home</description>
<link>http://kylin.apache.org/</link>
<atom:link href="http://kylin.apache.org/feed.xml" rel="self" type="application/rss+xml"/>
- <pubDate>Thu, 01 Aug 2019 06:52:56 -0700</pubDate>
- <lastBuildDate>Thu, 01 Aug 2019 06:52:56 -0700</lastBuildDate>
+ <pubDate>Thu, 08 Aug 2019 06:59:20 -0700</pubDate>
+ <lastBuildDate>Thu, 08 Aug 2019 06:59:20 -0700</lastBuildDate>
<generator>Jekyll v2.5.3</generator>
<item>
@@ -1191,114 +1191,114 @@ Security: (depend on your security setti
</item>
<item>
- <title>Apache Kylin v3.0.0-alpha åå¸</title>
- <description><p>è¿æ¥ Apache Kylin 社åºå¾é«å
´å°å®£å¸ï¼Apache Kylin v3.0.0-alpha æ£å¼åå¸ã</p>
+ <title>Apache Kylin v3.0.0-alpha Release Announcement</title>
+ <description><p>The Apache Kylin community is pleased to announce the release of Apache Kylin v3.0.0-alpha.</p>
-<p>Apache Kylin æ¯ä¸ä¸ªå¼æºçåå¸å¼åæå¼æï¼æ¨å¨ä¸ºæ大æ°æ®éæä¾ SQL æ¥å£åå¤ç»´åæï¼OLAPï¼çè½åã</p>
+<p>Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Big Data supporting extremely large datasets.</p>
-<p>è¿æ¯ Kylin ä¸ä¸ä»£ v3.x ç第ä¸ä¸ªåå¸çæ¬ï¼ç¨äºæ©æé¢è§ï¼ä¸»è¦çåè½æ¯å®æ¶ ï¼Real-timeï¼ OLAPãå®æ´çæ¹å¨å表请åè§<a href="/docs/release_notes.html">release notes</a>ï¼è¿éæä¸äºä¸»è¦æ¹è¿å说æã</p>
+<p>This is the first release of the new generation v3.x, the main feature introduced is the Real-time OLAP. All of the changes can be found in the <a href="/docs/release_notes.html">release notes</a>. Here we just highlight the main features.</p>
-<h1 id="section">éè¦æ°åè½</h1>
+<h1 id="important-features">Important features</h1>
-<h3 id="kylin-3654----olap">KYLIN-3654 - å®æ¶ OLAP</h3>
-<p>éçå¼å
¥æ°ç real-time receiver å coordinator ç»ä»¶ï¼Kylin è½å¤å®ç°æ¯«ç§çº§å«çæ°æ®åå¤å»¶è¿ï¼æ°æ®æºæ¥èªæµå¼æ°æ®å¦ Apache Kafkaãè¿æå³çï¼ä» v3.0 å¼å§ï¼Kylin æ¢è½å¤æ¯æåå²æ¹éæ°æ®ç OLAPï¼ä¹æ¯æ对æµå¼æ°æ®çåå®æ¶ï¼Near real-timeï¼ä»¥åå®å
¨å®æ¶(real-time)åæãç¨æ·å¯ä»¥ä½¿ç¨ä¸ä¸ª OLAP å¹³å°æ¥æå¡ä¸åç使ç¨åºæ¯ãæ¤æ¹æ¡å·²ç»å¨æ©æç¨æ·å¦ eBay å¾å°é¨ç½²åéªè¯ãå
³äºå¦ä½ä½¿ç¨æ¤åè½ï¼è¯·åè<a href="/docs30/tutorial/realtime_olap.html">æ¤æ
ç¨</a>ã</p>
+<h3 id="kylin-3654---real-time-olap">KYLIN-3654 - Real-time OLAP</h3>
+<p>With the newly introduced Kylin real-time receiver and coordinator components, Kylin can implement a millisecond-level data preparation delay for streaming data from sources like Apache Kafka. This means since v3.0 on, Kylin can support sub-second level OLAP over historical batch data, near real-time streaming as well as real-time streaming. The user can use one OLAP platform to serve different scenarios. This solution has been deployed and verified in early adopters like eBay since 2018. For how to enable it, please refer to <a href="/docs30/tutorial/realtime_olap.html">this tutorial</a>.</p>
-<h3 id="kylin-3795----apache-livy--spark-">KYLIN-3795 - éè¿ Apache Livy é交 Spark ä»»å¡</h3>
-<p>è¿ä¸ªåè½å
许管çå为 Kylin é
ç½®ä½¿ç¨ Apache Livy (incubating) æ¥å®æä»»å¡çé交ãSpark ä½ä¸çæ交éè¿ Livy ç REST API æ¥æ交ï¼èæ éå¨æ¬å°å¯å¨ Spark Driver è¿ç¨ï¼ä»èæ¹ä¾¿å¯¹ Spark èµæºç管ççæ§ï¼åæ¶ä¹éä½å¯¹ Kylin ä»»å¡è¿ç¨æå¨èç¹çååã</p>
+<h3 id="kylin-3795---submit-spark-jobs-via-apache-livy">KYLIN-3795 - Submit Spark jobs via Apache Livy</h3>
+<p>This feature allows the administrator to configure Kylin to integrate with Apache Livy (incubating) for Spark job submissions. The Spark job is submitted to the Livy Server through Livyâs REST API, instead of starting the Spark Driver process in local, which facilitates the management and monitoring of the Spark resources, and also releases the pressure of the nodes where the Kylin job server is running.</p>
-<h3 id="kylin-3820----curator-">KYLIN-3820 - åºäº Curator çä»»å¡èç¹åé
åæå¡åç°</h3>
-<p>æ°å¢ä¸ç§åºäºApache Zookeeper å Curatorä½ä¸è°åº¦å¨ï¼å¯ä»¥èªå¨åç° Kylin èç¹ï¼å¹¶èªå¨åé
ä¸ä¸ªèç¹æ¥è¿è¡ä»»å¡ç管ç以åæ
éæ¢å¤ãæäºè¿ä¸ªåè½åï¼ç®¡çåå¯ä»¥æ´å 容æå°é¨ç½²åæ©å± Kylin èç¹ï¼èä¸åéè¦å¨ <code class="highlighter-rouge">kylin.properties</code> ä¸é
ç½®æ¯ä¸ª Kylin èç¹çå°å并éå¯ Kylin 以使ä¹çæã</p>
+<h3 id="kylin-3820---a-curator-based-job-scheduler">KYLIN-3820 - A curator-based job scheduler</h3>
+<p>A new job scheduler is added to automatically discover the Kylin nodes and do an automatic leader selection among them (only the leader will submit jobs). With this feature, you can easily deploy and scale out Kylin nodes without manually update the node address in <code class="highlighter-rouge">kylin.properties</code> and restart Kylin to take effective.</p>
-<h1 id="section-1">å
¶å®æ¹è¿</h1>
+<h1 id="other-enhancements">Other enhancements</h1>
-<h3 id="kylin-3716---fastthreadlocal--threadlocal">KYLIN-3716 - FastThreadLocal æ¿æ¢ ThreadLocal</h3>
-<p>ä½¿ç¨ Netty ä¸ç FastThreadLocal æ¿ä»£ JDK åçç ThreadLocalï¼å¯ä»¥ä¸å®ç¨åº¦ä¸æå Kylin å¨é«å¹¶åä¸çæ§è½ã</p>
+<h3 id="kylin-3716---fastthreadlocal-replaces-threadlocal">KYLIN-3716 - FastThreadLocal replaces ThreadLocal</h3>
+<p>Using FastThreadLocal instead of ThreadLocal can improve Kylinâs overall performance to some extent.</p>
<h3 id="kylin-3867---enable-jdbc-to-use-key-store--trust-store-for-https-connection">KYLIN-3867 - Enable JDBC to use key store &amp; trust store for https connection</h3>
-<p>éè¿ä½¿ç¨HTTPSï¼ä¿æ¤äºJDBC使ç¨ç身份éªè¯ä¿¡æ¯ï¼ä½¿å¾Kylinæ´å å®å
¨</p>
+<p>By using HTTPS, the authentication information used by JDBC is protected, making Kylin more secure.</p>
<h3 id="kylin-3905---enable-shrunken-dictionary-default">KYLIN-3905 - Enable shrunken dictionary default</h3>
-<p>é»è®¤å¼å¯ shrunken dictionaryï¼é对é«åºç»´è¿è¡ç²¾ç¡®å»éçåºæ¯ï¼å¯ä»¥æ¾èåå°æ建ç¨æ¶ã</p>
+<p>By default, the shrunken dictionary is enabled, and the precise counting scene for high cardinal dimensions can significantly reduce the build time.</p>
<h3 id="kylin-3839---storage-clean-up-after-the-refreshing-and-deleting-a-segment">KYLIN-3839 - Storage clean up after the refreshing and deleting a segment</h3>
-<p>æ´å åæ¶å°æ¸
é¤ä¸å¿
è¦çæ°æ®æ件</p>
+<p>Clear unnecessary data files in a timely manner</p>
-<p><strong>ä¸è½½</strong></p>
+<p><strong>Download</strong></p>
-<p>è¦ä¸è½½Apache Kylin æºä»£ç æäºè¿å¶å
ï¼è¯·è®¿é®<a href="/download">ä¸è½½é¡µé¢</a> page.</p>
+<p>To download Apache Kylin v3.0.0-alpha source code or binary package, visit the <a href="http://kylin.apache.org/download">download</a> page.</p>
-<p><strong>å级</strong></p>
+<p><strong>Upgrade</strong></p>
-<p>åè<a href="/docs/howto/howto_upgrade.html">å级æå</a>.</p>
+<p>Follow the <a href="/docs/howto/howto_upgrade.html">upgrade guide</a>.</p>
-<p><strong>åé¦</strong></p>
+<p><strong>Feedback</strong></p>
-<p>å¦ææ¨éå°é®é¢æçé®ï¼è¯·åéé®ä»¶è³ Apache Kylin dev æ user é®ä»¶å表ï¼dev@kylin.apache.orgï¼user@kylin.apache.org; å¨åéä¹åï¼è¯·ç¡®ä¿æ¨å·²éè¿åéçµåé®ä»¶è³ dev-subscribe@kylin.apache.org æ user-subscribe@kylin.apache.org 订é
äºé®ä»¶å表ã</p>
+<p>If you face issue or question, please send mail to Apache Kylin dev or user mailing list:Â dev@kylin.apache.org , user@kylin.apache.org; Before sending, please make sure you have subscribed the mailing list by dropping an email to dev-subscribe@kylin.apache.org or user-subscribe@kylin.apache.org.</p>
-<p><em>é常æè°¢ææè´¡ç®Apache Kylinçæå!</em></p>
+<p><em>Great thanks to everyone who contributed!</em></p>
</description>
<pubDate>Fri, 19 Apr 2019 13:00:00 -0700</pubDate>
- <link>http://kylin.apache.org/cn_blog/2019/04/19/release-v3.0.0-alpha/</link>
- <guid isPermaLink="true">http://kylin.apache.org/cn_blog/2019/04/19/release-v3.0.0-alpha/</guid>
+ <link>http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</link>
+ <guid isPermaLink="true">http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</guid>
- <category>cn_blog</category>
+ <category>blog</category>
</item>
<item>
- <title>Apache Kylin v3.0.0-alpha Release Announcement</title>
- <description><p>The Apache Kylin community is pleased to announce the release of Apache Kylin v3.0.0-alpha.</p>
+ <title>Apache Kylin v3.0.0-alpha åå¸</title>
+ <description><p>è¿æ¥ Apache Kylin 社åºå¾é«å
´å°å®£å¸ï¼Apache Kylin v3.0.0-alpha æ£å¼åå¸ã</p>
-<p>Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Big Data supporting extremely large datasets.</p>
+<p>Apache Kylin æ¯ä¸ä¸ªå¼æºçåå¸å¼åæå¼æï¼æ¨å¨ä¸ºæ大æ°æ®éæä¾ SQL æ¥å£åå¤ç»´åæï¼OLAPï¼çè½åã</p>
-<p>This is the first release of the new generation v3.x, the main feature introduced is the Real-time OLAP. All of the changes can be found in the <a href="/docs/release_notes.html">release notes</a>. Here we just highlight the main features.</p>
+<p>è¿æ¯ Kylin ä¸ä¸ä»£ v3.x ç第ä¸ä¸ªåå¸çæ¬ï¼ç¨äºæ©æé¢è§ï¼ä¸»è¦çåè½æ¯å®æ¶ ï¼Real-timeï¼ OLAPãå®æ´çæ¹å¨å表请åè§<a href="/docs/release_notes.html">release notes</a>ï¼è¿éæä¸äºä¸»è¦æ¹è¿å说æã</p>
-<h1 id="important-features">Important features</h1>
+<h1 id="section">éè¦æ°åè½</h1>
-<h3 id="kylin-3654---real-time-olap">KYLIN-3654 - Real-time OLAP</h3>
-<p>With the newly introduced Kylin real-time receiver and coordinator components, Kylin can implement a millisecond-level data preparation delay for streaming data from sources like Apache Kafka. This means since v3.0 on, Kylin can support sub-second level OLAP over historical batch data, near real-time streaming as well as real-time streaming. The user can use one OLAP platform to serve different scenarios. This solution has been deployed and verified in early adopters like eBay since 2018. For how to enable it, please refer to <a href="/docs30/tutorial/realtime_olap.html">this tutorial</a>.</p>
+<h3 id="kylin-3654----olap">KYLIN-3654 - å®æ¶ OLAP</h3>
+<p>éçå¼å
¥æ°ç real-time receiver å coordinator ç»ä»¶ï¼Kylin è½å¤å®ç°æ¯«ç§çº§å«çæ°æ®åå¤å»¶è¿ï¼æ°æ®æºæ¥èªæµå¼æ°æ®å¦ Apache Kafkaãè¿æå³çï¼ä» v3.0 å¼å§ï¼Kylin æ¢è½å¤æ¯æåå²æ¹éæ°æ®ç OLAPï¼ä¹æ¯æ对æµå¼æ°æ®çåå®æ¶ï¼Near real-timeï¼ä»¥åå®å
¨å®æ¶(real-time)åæãç¨æ·å¯ä»¥ä½¿ç¨ä¸ä¸ª OLAP å¹³å°æ¥æå¡ä¸åç使ç¨åºæ¯ãæ¤æ¹æ¡å·²ç»å¨æ©æç¨æ·å¦ eBay å¾å°é¨ç½²åéªè¯ãå
³äºå¦ä½ä½¿ç¨æ¤åè½ï¼è¯·åè<a href="/docs30/tutorial/realtime_olap.html">æ¤æ
ç¨</a>ã</p>
-<h3 id="kylin-3795---submit-spark-jobs-via-apache-livy">KYLIN-3795 - Submit Spark jobs via Apache Livy</h3>
-<p>This feature allows the administrator to configure Kylin to integrate with Apache Livy (incubating) for Spark job submissions. The Spark job is submitted to the Livy Server through Livyâs REST API, instead of starting the Spark Driver process in local, which facilitates the management and monitoring of the Spark resources, and also releases the pressure of the nodes where the Kylin job server is running.</p>
+<h3 id="kylin-3795----apache-livy--spark-">KYLIN-3795 - éè¿ Apache Livy é交 Spark ä»»å¡</h3>
+<p>è¿ä¸ªåè½å
许管çå为 Kylin é
ç½®ä½¿ç¨ Apache Livy (incubating) æ¥å®æä»»å¡çé交ãSpark ä½ä¸çæ交éè¿ Livy ç REST API æ¥æ交ï¼èæ éå¨æ¬å°å¯å¨ Spark Driver è¿ç¨ï¼ä»èæ¹ä¾¿å¯¹ Spark èµæºç管ççæ§ï¼åæ¶ä¹éä½å¯¹ Kylin ä»»å¡è¿ç¨æå¨èç¹çååã</p>
-<h3 id="kylin-3820---a-curator-based-job-scheduler">KYLIN-3820 - A curator-based job scheduler</h3>
-<p>A new job scheduler is added to automatically discover the Kylin nodes and do an automatic leader selection among them (only the leader will submit jobs). With this feature, you can easily deploy and scale out Kylin nodes without manually update the node address in <code class="highlighter-rouge">kylin.properties</code> and restart Kylin to take effective.</p>
+<h3 id="kylin-3820----curator-">KYLIN-3820 - åºäº Curator çä»»å¡èç¹åé
åæå¡åç°</h3>
+<p>æ°å¢ä¸ç§åºäºApache Zookeeper å Curatorä½ä¸è°åº¦å¨ï¼å¯ä»¥èªå¨åç° Kylin èç¹ï¼å¹¶èªå¨åé
ä¸ä¸ªèç¹æ¥è¿è¡ä»»å¡ç管ç以åæ
éæ¢å¤ãæäºè¿ä¸ªåè½åï¼ç®¡çåå¯ä»¥æ´å 容æå°é¨ç½²åæ©å± Kylin èç¹ï¼èä¸åéè¦å¨ <code class="highlighter-rouge">kylin.properties</code> ä¸é
ç½®æ¯ä¸ª Kylin èç¹çå°å并éå¯ Kylin 以使ä¹çæã</p>
-<h1 id="other-enhancements">Other enhancements</h1>
+<h1 id="section-1">å
¶å®æ¹è¿</h1>
-<h3 id="kylin-3716---fastthreadlocal-replaces-threadlocal">KYLIN-3716 - FastThreadLocal replaces ThreadLocal</h3>
-<p>Using FastThreadLocal instead of ThreadLocal can improve Kylinâs overall performance to some extent.</p>
+<h3 id="kylin-3716---fastthreadlocal--threadlocal">KYLIN-3716 - FastThreadLocal æ¿æ¢ ThreadLocal</h3>
+<p>ä½¿ç¨ Netty ä¸ç FastThreadLocal æ¿ä»£ JDK åçç ThreadLocalï¼å¯ä»¥ä¸å®ç¨åº¦ä¸æå Kylin å¨é«å¹¶åä¸çæ§è½ã</p>
<h3 id="kylin-3867---enable-jdbc-to-use-key-store--trust-store-for-https-connection">KYLIN-3867 - Enable JDBC to use key store &amp; trust store for https connection</h3>
-<p>By using HTTPS, the authentication information used by JDBC is protected, making Kylin more secure.</p>
+<p>éè¿ä½¿ç¨HTTPSï¼ä¿æ¤äºJDBC使ç¨ç身份éªè¯ä¿¡æ¯ï¼ä½¿å¾Kylinæ´å å®å
¨</p>
<h3 id="kylin-3905---enable-shrunken-dictionary-default">KYLIN-3905 - Enable shrunken dictionary default</h3>
-<p>By default, the shrunken dictionary is enabled, and the precise counting scene for high cardinal dimensions can significantly reduce the build time.</p>
+<p>é»è®¤å¼å¯ shrunken dictionaryï¼é对é«åºç»´è¿è¡ç²¾ç¡®å»éçåºæ¯ï¼å¯ä»¥æ¾èåå°æ建ç¨æ¶ã</p>
<h3 id="kylin-3839---storage-clean-up-after-the-refreshing-and-deleting-a-segment">KYLIN-3839 - Storage clean up after the refreshing and deleting a segment</h3>
-<p>Clear unnecessary data files in a timely manner</p>
+<p>æ´å åæ¶å°æ¸
é¤ä¸å¿
è¦çæ°æ®æ件</p>
-<p><strong>Download</strong></p>
+<p><strong>ä¸è½½</strong></p>
-<p>To download Apache Kylin v3.0.0-alpha source code or binary package, visit the <a href="http://kylin.apache.org/download">download</a> page.</p>
+<p>è¦ä¸è½½Apache Kylin æºä»£ç æäºè¿å¶å
ï¼è¯·è®¿é®<a href="/download">ä¸è½½é¡µé¢</a> page.</p>
-<p><strong>Upgrade</strong></p>
+<p><strong>å级</strong></p>
-<p>Follow the <a href="/docs/howto/howto_upgrade.html">upgrade guide</a>.</p>
+<p>åè<a href="/docs/howto/howto_upgrade.html">å级æå</a>.</p>
-<p><strong>Feedback</strong></p>
+<p><strong>åé¦</strong></p>
-<p>If you face issue or question, please send mail to Apache Kylin dev or user mailing list:Â dev@kylin.apache.org , user@kylin.apache.org; Before sending, please make sure you have subscribed the mailing list by dropping an email to dev-subscribe@kylin.apache.org or user-subscribe@kylin.apache.org.</p>
+<p>å¦ææ¨éå°é®é¢æçé®ï¼è¯·åéé®ä»¶è³ Apache Kylin dev æ user é®ä»¶å表ï¼dev@kylin.apache.orgï¼user@kylin.apache.org; å¨åéä¹åï¼è¯·ç¡®ä¿æ¨å·²éè¿åéçµåé®ä»¶è³ dev-subscribe@kylin.apache.org æ user-subscribe@kylin.apache.org 订é
äºé®ä»¶å表ã</p>
-<p><em>Great thanks to everyone who contributed!</em></p>
+<p><em>é常æè°¢ææè´¡ç®Apache Kylinçæå!</em></p>
</description>
<pubDate>Fri, 19 Apr 2019 13:00:00 -0700</pubDate>
- <link>http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</link>
- <guid isPermaLink="true">http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</guid>
+ <link>http://kylin.apache.org/cn_blog/2019/04/19/release-v3.0.0-alpha/</link>
+ <guid isPermaLink="true">http://kylin.apache.org/cn_blog/2019/04/19/release-v3.0.0-alpha/</guid>
- <category>blog</category>
+ <category>cn_blog</category>
</item>
@@ -1455,84 +1455,6 @@ The checkpoint info is the smallest part
</item>
<item>
- <title>Apache Kylin v2.6.0 Release Announcement</title>
- <description><p>The Apache Kylin community is pleased to announce the release of Apache Kylin v2.6.0.</p>
-
-<p>Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Big Data supporting extremely large datasets.</p>
-
-<p>This is a major release after 2.5.0, including many enhancements. All of the changes can be found in the <a href="https://kylin.apache.org/docs/release_notes.html">release notes</a>. Here just highlight the major ones:</p>
-
-<h3 id="sdk-for-jdbc-sources">SDK for JDBC sources</h3>
-<p>Apache Kylin has already supported several data sources like Amazon Redshift, SQL Server through JDBC. <br />
-To help developers handle SQL dialect differences and easily implement a new data source through JDBC, Kylin provides a new data source SDK with APIs for:<br />
-* Synchronize metadata and data from JDBC source<br />
-* Build cube from JDBC source<br />
-* Query pushdown to JDBC source engine when cube is unmatched</p>
-
-<p>Check KYLIN-3552 for more.</p>
-
-<h3 id="memcached-as-distributed-cache">Memcached as distributed cache</h3>
-<p>In the past, query caches are not efficiently used in Kylin due to two aspects: aggressive cache expiration strategy and local cache. <br />
-Because of the aggressive cache expiration strategy, useful caches are often cleaned up unnecessarily. <br />
-Because query caches are stored in local servers, they cannot be shared between servers. <br />
-And because of the size limitation of local cache, not all useful query results can be cached.</p>
-
-<p>To deal with these shortcomings, we change the query cache expiration strategy by signature checking and introduce the memcached as Kylinâs distributed cache so that Kylin servers are able to share cache between servers. <br />
-And itâs easy to add memcached servers to scale out distributed cache. With enough memcached servers, we can cached things as much as possible. <br />
-Then we also introduce segment level query cache which can not only speed up query but also reduce the rpcs to HBase. <br />
-The related tasks are KYLIN-2895, KYLIN-2894, KYLIN-2896, KYLIN-2897, KYLIN-2898, KYLIN-2899.</p>
-
-<h3 id="forkjoinpool-for-fast-cubing">ForkJoinPool for fast cubing</h3>
-<p>In the past, fast cubing uses split threads, task threads and main thread to do the cube building, there is complex join and error handling logic.</p>
-
-<p>The new implement leverages the ForkJoinPool from JDK, the event split logic is handled in<br />
-main thread. Cuboid task and sub-tasks are handled in fork join pool, cube results are collected<br />
-async and can be write to output earlier. Check KYLIN-2932 for more.</p>
-
-<h3 id="improve-hllcounter-performance">Improve HLLCounter performance</h3>
-<p>In the past, the way to create HLLCounter and to compute harmonic mean are not efficient.</p>
-
-<p>The new implement improve the HLLCounter creation by copy register from another HLLCounter instead of merge. To compute harmonic mean in the HLLCSnapshot, it does the enhancement by <br />
-* using table to cache all 1/2^r without computing on the fly<br />
-* remove floating addition by using integer addition in the bigger loop<br />
-* remove branch, e.g. neednât checking whether registers[i] is zero or not, although this is minor improvement.</p>
-
-<p>Check KYLIN-3656 for more.</p>
-
-<h3 id="improve-cuboid-recommendation-algorithm">Improve Cuboid Recommendation Algorithm</h3>
-<p>In the past, to add cuboids which are not prebuilt, the cube planner turns to mandatory cuboids which are selected if its rollup row count is above some threshold. <br />
-There are two shortcomings:<br />
-* The way to estimate the rollup row count is not good<br />
-* Itâs hard to determine the threshold of rollup row count for recommending mandatory cuboids</p>
-
-<p>The new implement improves the way to estimate the row count of un-prebuilt cuboids by rollup ratio rather than exact rollup row count. <br />
-With better estimated row counts for un-prebuilt cuboids, the cost-based cube planner algorithm will decide which cuboid to be built or not and the threshold for previous mandatory cuboids is not needed. <br />
-By this improvement, we donât need the threshold for mandatory cuboids recommendation, and mandatory cuboids can only be manually set and will not be recommended. Check KYLIN-3540 for more.</p>
-
-<p><strong>Download</strong></p>
-
-<p>To download Apache Kylin v2.6.0 source code or binary package, visit the <a href="http://kylin.apache.org/download">download</a> page.</p>
-
-<p><strong>Upgrade</strong></p>
-
-<p>Follow the <a href="/docs/howto/howto_upgrade.html">upgrade guide</a>.</p>
-
-<p><strong>Feedback</strong></p>
-
-<p>If you face issue or question, please send mail to Apache Kylin dev or user mailing list:Â dev@kylin.apache.org , user@kylin.apache.org; Before sending, please make sure you have subscribed the mailing list by dropping an email to dev-subscribe@kylin.apache.org or user-subscribe@kylin.apache.org.</p>
-
-<p><em>Great thanks to everyone who contributed!</em></p>
-</description>
- <pubDate>Fri, 18 Jan 2019 12:00:00 -0800</pubDate>
- <link>http://kylin.apache.org/blog/2019/01/18/release-v2.6.0/</link>
- <guid isPermaLink="true">http://kylin.apache.org/blog/2019/01/18/release-v2.6.0/</guid>
-
-
- <category>blog</category>
-
- </item>
-
- <item>
<title>Apache Kylin v2.6.0 æ£å¼åå¸</title>
<description><p>è¿æ¥Apache Kylin 社åºå¾é«å
´å°å®£å¸ï¼Apache Kylin 2.6.0 æ£å¼åå¸ã</p>
@@ -1611,5 +1533,83 @@ By this improvement, we donât need
</item>
+ <item>
+ <title>Apache Kylin v2.6.0 Release Announcement</title>
+ <description><p>The Apache Kylin community is pleased to announce the release of Apache Kylin v2.6.0.</p>
+
+<p>Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Big Data supporting extremely large datasets.</p>
+
+<p>This is a major release after 2.5.0, including many enhancements. All of the changes can be found in the <a href="https://kylin.apache.org/docs/release_notes.html">release notes</a>. Here just highlight the major ones:</p>
+
+<h3 id="sdk-for-jdbc-sources">SDK for JDBC sources</h3>
+<p>Apache Kylin has already supported several data sources like Amazon Redshift, SQL Server through JDBC. <br />
+To help developers handle SQL dialect differences and easily implement a new data source through JDBC, Kylin provides a new data source SDK with APIs for:<br />
+* Synchronize metadata and data from JDBC source<br />
+* Build cube from JDBC source<br />
+* Query pushdown to JDBC source engine when cube is unmatched</p>
+
+<p>Check KYLIN-3552 for more.</p>
+
+<h3 id="memcached-as-distributed-cache">Memcached as distributed cache</h3>
+<p>In the past, query caches are not efficiently used in Kylin due to two aspects: aggressive cache expiration strategy and local cache. <br />
+Because of the aggressive cache expiration strategy, useful caches are often cleaned up unnecessarily. <br />
+Because query caches are stored in local servers, they cannot be shared between servers. <br />
+And because of the size limitation of local cache, not all useful query results can be cached.</p>
+
+<p>To deal with these shortcomings, we change the query cache expiration strategy by signature checking and introduce the memcached as Kylinâs distributed cache so that Kylin servers are able to share cache between servers. <br />
+And itâs easy to add memcached servers to scale out distributed cache. With enough memcached servers, we can cached things as much as possible. <br />
+Then we also introduce segment level query cache which can not only speed up query but also reduce the rpcs to HBase. <br />
+The related tasks are KYLIN-2895, KYLIN-2894, KYLIN-2896, KYLIN-2897, KYLIN-2898, KYLIN-2899.</p>
+
+<h3 id="forkjoinpool-for-fast-cubing">ForkJoinPool for fast cubing</h3>
+<p>In the past, fast cubing uses split threads, task threads and main thread to do the cube building, there is complex join and error handling logic.</p>
+
+<p>The new implement leverages the ForkJoinPool from JDK, the event split logic is handled in<br />
+main thread. Cuboid task and sub-tasks are handled in fork join pool, cube results are collected<br />
+async and can be write to output earlier. Check KYLIN-2932 for more.</p>
+
+<h3 id="improve-hllcounter-performance">Improve HLLCounter performance</h3>
+<p>In the past, the way to create HLLCounter and to compute harmonic mean are not efficient.</p>
+
+<p>The new implement improve the HLLCounter creation by copy register from another HLLCounter instead of merge. To compute harmonic mean in the HLLCSnapshot, it does the enhancement by <br />
+* using table to cache all 1/2^r without computing on the fly<br />
+* remove floating addition by using integer addition in the bigger loop<br />
+* remove branch, e.g. neednât checking whether registers[i] is zero or not, although this is minor improvement.</p>
+
+<p>Check KYLIN-3656 for more.</p>
+
+<h3 id="improve-cuboid-recommendation-algorithm">Improve Cuboid Recommendation Algorithm</h3>
+<p>In the past, to add cuboids which are not prebuilt, the cube planner turns to mandatory cuboids which are selected if its rollup row count is above some threshold. <br />
+There are two shortcomings:<br />
+* The way to estimate the rollup row count is not good<br />
+* Itâs hard to determine the threshold of rollup row count for recommending mandatory cuboids</p>
+
+<p>The new implement improves the way to estimate the row count of un-prebuilt cuboids by rollup ratio rather than exact rollup row count. <br />
+With better estimated row counts for un-prebuilt cuboids, the cost-based cube planner algorithm will decide which cuboid to be built or not and the threshold for previous mandatory cuboids is not needed. <br />
+By this improvement, we donât need the threshold for mandatory cuboids recommendation, and mandatory cuboids can only be manually set and will not be recommended. Check KYLIN-3540 for more.</p>
+
+<p><strong>Download</strong></p>
+
+<p>To download Apache Kylin v2.6.0 source code or binary package, visit the <a href="http://kylin.apache.org/download">download</a> page.</p>
+
+<p><strong>Upgrade</strong></p>
+
+<p>Follow the <a href="/docs/howto/howto_upgrade.html">upgrade guide</a>.</p>
+
+<p><strong>Feedback</strong></p>
+
+<p>If you face issue or question, please send mail to Apache Kylin dev or user mailing list:Â dev@kylin.apache.org , user@kylin.apache.org; Before sending, please make sure you have subscribed the mailing list by dropping an email to dev-subscribe@kylin.apache.org or user-subscribe@kylin.apache.org.</p>
+
+<p><em>Great thanks to everyone who contributed!</em></p>
+</description>
+ <pubDate>Fri, 18 Jan 2019 12:00:00 -0800</pubDate>
+ <link>http://kylin.apache.org/blog/2019/01/18/release-v2.6.0/</link>
+ <guid isPermaLink="true">http://kylin.apache.org/blog/2019/01/18/release-v2.6.0/</guid>
+
+
+ <category>blog</category>
+
+ </item>
+
</channel>
</rss>
Added: kylin/site/images/Hive-Global-Dictionary/cube-level-config.png
URL: http://svn.apache.org/viewvc/kylin/site/images/Hive-Global-Dictionary/cube-level-config.png?rev=1864708&view=auto
==============================================================================
Binary file - no diff available.
Propchange: kylin/site/images/Hive-Global-Dictionary/cube-level-config.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: kylin/site/images/Hive-Global-Dictionary/hive-global-dict-table.png
URL: http://svn.apache.org/viewvc/kylin/site/images/Hive-Global-Dictionary/hive-global-dict-table.png?rev=1864708&view=auto
==============================================================================
Binary file - no diff available.
Propchange: kylin/site/images/Hive-Global-Dictionary/hive-global-dict-table.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: kylin/site/images/Hive-Global-Dictionary/set-hive-dict-column.png
URL: http://svn.apache.org/viewvc/kylin/site/images/Hive-Global-Dictionary/set-hive-dict-column.png?rev=1864708&view=auto
==============================================================================
Binary file - no diff available.
Propchange: kylin/site/images/Hive-Global-Dictionary/set-hive-dict-column.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: kylin/site/images/Hive-Global-Dictionary/three-added-steps.png
URL: http://svn.apache.org/viewvc/kylin/site/images/Hive-Global-Dictionary/three-added-steps.png?rev=1864708&view=auto
==============================================================================
Binary file - no diff available.
Propchange: kylin/site/images/Hive-Global-Dictionary/three-added-steps.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream