You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kylin.apache.org by li...@apache.org on 2019/06/06 08:35:44 UTC
svn commit: r1860706 [2/2] - in /kylin/site: blog/2019/05/29/
blog/2019/05/29/kylin-2.4.1-to-2.6.1/
blog/2019/05/29/kylin-2.4.1-to-2.6.1/index.html blog/index.html feed.xml
Modified: kylin/site/feed.xml
URL: http://svn.apache.org/viewvc/kylin/site/feed.xml?rev=1860706&r1=1860705&r2=1860706&view=diff
==============================================================================
--- kylin/site/feed.xml (original)
+++ kylin/site/feed.xml Thu Jun 6 08:35:44 2019
@@ -19,11 +19,84 @@
<description>Apache Kylin Home</description>
<link>http://kylin.apache.org/</link>
<atom:link href="http://kylin.apache.org/feed.xml" rel="self" type="application/rss+xml"/>
- <pubDate>Mon, 03 Jun 2019 06:59:28 -0700</pubDate>
- <lastBuildDate>Mon, 03 Jun 2019 06:59:28 -0700</lastBuildDate>
+ <pubDate>Thu, 06 Jun 2019 01:23:11 -0700</pubDate>
+ <lastBuildDate>Thu, 06 Jun 2019 01:23:11 -0700</lastBuildDate>
<generator>Jekyll v2.5.3</generator>
<item>
+ <title>Kylin upgrade 2.4.1 to 2.6.1</title>
+ <description><p>At Telecoming we started using Kylin as our main analytics database in a new Business Intelligence project that started by the end of 2017. We moved from a custom report engine based in MySQL and later in AWS Redshift into a fully Hadoop based solution with Kylin as last step prior reporting generation. We started with Kylin 2.2.1, migrated to 2.4.1 by mid 2018 and moved to 2.6.1 last month. Kylin 2.2.1 and our first release but had some annoying bugs and stability problems. It was not only due to less mature release but also because didnât have experience running Kylin. Some design changes at cube level and optimizations (thank you, Alberto Ramon) helped us a lot in order to improve performance and stability but there were still some issues present that has been solved in later versions.</p>
+
+<p>This is our experience.</p>
+
+<h3 id="what-we-were-looking-for-with-this-upgrade">What we were looking for with this upgrade:</h3>
+
+<p>1) Bug fixing. In particular one bug related with views when creating intermediate hive tables.</p>
+
+<p>2) Performance enhancements. This is always welcomed.</p>
+
+<p>3) Less dependency at HBase level. HBase is our main source of problems. In part, due to EMR (AWS) distribution that relies in S3 as main storage (now we are in HDP). Table metadata is frequenly accesed and response is not always good. So, we wanted to avoid Hbase storage for metadata table and move to mysql via jdbc.</p>
+
+<h3 id="upgrade-plan">Upgrade plan:</h3>
+
+<p>1) Minimize downtime. Since our users do intensive use of reporting system, itâs imperative to minimize Kylin downtime.</p>
+
+<p>2) Easy rollback. Just in case of a unsuccessful upgrade, we need an easy way to rollback.</p>
+
+<h3 id="what-we-did">What we did:</h3>
+
+<p>1) We prepared a new AWS instance with Kylin 2.6.1, already configured and tuned. Of course, previously we had run a for a few weeks a Kylin 2.6.1 prototype and tweaked all config files in order to run properly. This new instance was stopped when we started the migration. It also had a brand new local mysql 5.7 for metadata storage instead of relying on HBase.</p>
+
+<p>2) We stopped all cube buildings on Kylin 2.4.1. We waited until those that were in progress had finished. So, at query level everything was working although no more data were added.</p>
+
+<p>3) We performed a full backup of Kylin metadata (metastore.sh backup) when all builds had finished. It took only 2 minutes.</p>
+
+<p>4) We performed an Hbase snapshot over all Hbase tables. So, rolling back in case of a total failure to previous version was an easy task: restore metadata + clone snapshot into new table (removing upgraded table). It took only a few seconds since you can run all snapshot commands in a single sentence.</p>
+
+<p>5) We performed a metadata restore on our new Kylin 2.6.1 instance (metastore.sh restore). In this way, all metadata were automatically migrated from Hbase based storage to jdbc based storage in a few minutes.</p>
+
+<p>6) We upgraded coprocessor from 2.6.1 (with kylin.sh org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI default all).</p>
+
+<p>7) We started new kylin version and checked that everything was in place. We could query all cubes and got proper responses.</p>
+
+<p>8) After all test were finished, we pointed our DNS entries to new kylin 2.6.1 instance and resumed the building processes from it. Of course, kylin 2.4.1 was shutdown. Some days later, we removed hbase snapshots in order to free up storage space.</p>
+
+<p>As you can imagine, our downtime was near zero at query level since all kylin queries were run at 2.4.1 until 2.6.1 was ready to serve requests.</p>
+
+<h3 id="rollback-plan-not-needed">Rollback plan (not needed)</h3>
+
+<p>Rollback plan was very simple. In case of problem, we would switch to previous version and we would restore all Hbase tables from previously taken snapshots. This snapshots included Hbase metadata table, so, no need to restore metadata.</p>
+
+<h3 id="what-we-have-got">What we have got</h3>
+<ul>
+ <li>Kylin 2.6.1 is more stable than 2.4.1. Only problems we have found at server level has been due to memory issues at platform level. Now, many steps are spark based and take some extra memory when launching jobs. We solved easily by adjusting JVM (-Xmx, -Xms) parameters for kylin 2.6.1.</li>
+ <li>Cube builds are faster due to improvements at cube build level.</li>
+ <li>User interface is really fast. Previously browsing cube build list took up to 10 seconds. Now, it almost instantaneous.</li>
+ <li>Some bugs are not present anymore. Two of them were in particular very problematic:
+ <ul>
+ <li>Hive intermediate tables from views didnât add an uuid suffix. So, if the same table was used for other builds, it could be accidentally deleted by previous job at cleaning phase. So, build failed.</li>
+ <li>Param âkylin.job.cube-auto-ready-enabledâ was not working. So, after building a segment it was automatically enabled and not always we wanted this.</li>
+ </ul>
+ </li>
+</ul>
+
+<h3 id="final-thoughts">Final thoughts</h3>
+
+<p>In general, upgrade procedure has been easy and fast. However, a lot of previous work had to be done in order to test that upgrade would not be a problem. So, we built a parallel environment in order to check that all new functionalities were working and bugs were solved. We adapted our config files to new Kylin requirements, for example by replacing metadata storage from hbase to jdbc.<br />
+Upgrading a Kylin version to a new one is, in general, a simple task. However, itâs imperative to read carefully documentation because sometimes it require extra steps, and of course, check parameters in order to detect new features or changes that has to be taken in consideration in the new version. Having a backup and a rollback plan (tested) to come back as fast as possible is also very important.<br />
+From our users side, the most successful problem solved is the interface response. They are really happy how kylin interface is working right now. And from support side dealing with a MySQL database instead of an Hbase is also a great improvement. Editing a parameter inside a json on hbase can be hard since there are not many applications for editing a modifying a Hbase table.</p>
+
+</description>
+ <pubDate>Wed, 29 May 2019 05:00:00 -0700</pubDate>
+ <link>http://kylin.apache.org/blog/2019/05/29/kylin-2.4.1-to-2.6.1/</link>
+ <guid isPermaLink="true">http://kylin.apache.org/blog/2019/05/29/kylin-2.4.1-to-2.6.1/</guid>
+
+
+ <category>blog</category>
+
+ </item>
+
+ <item>
<title>Saiku + Kylin æ建å¤ç»´ OLAP å¹³å°</title>
<description><p>éçæ°æ®éçæ¿å¢ï¼ä¼ ç»ç OLTP å¹³å°å·²æ æ³æ»¡è¶³ç¨æ·çåæéæ±ãOLAP å¹³å°åºè¿èçï¼OLAP å¹³å°éå¸¸ç± OLAP Engine ä¸ç¨æ·æä½åæ页é¢ç»æãæ¬æå°ä¼ä»¥ Saiku + Kylin çç»åæ¹å¼è®²è§£å¤ç»´ OLAP å¹³å°çæ建ã</p>
@@ -169,55 +242,55 @@ Security: (depend on your security setti
</item>
<item>
- <title>Apache Kylin v3.0.0-alpha Release Announcement</title>
- <description><p>The Apache Kylin community is pleased to announce the release of Apache Kylin v3.0.0-alpha.</p>
+ <title>Apache Kylin v3.0.0-alpha åå¸</title>
+ <description><p>è¿æ¥ Apache Kylin 社åºå¾é«å
´å°å®£å¸ï¼Apache Kylin v3.0.0-alpha æ£å¼åå¸ã</p>
-<p>Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Big Data supporting extremely large datasets.</p>
+<p>Apache Kylin æ¯ä¸ä¸ªå¼æºçåå¸å¼åæå¼æï¼æ¨å¨ä¸ºæ大æ°æ®éæä¾ SQL æ¥å£åå¤ç»´åæï¼OLAPï¼çè½åã</p>
-<p>This is the first release of the new generation v3.x, the main feature introduced is the Real-time OLAP. All of the changes can be found in the <a href="/docs/release_notes.html">release notes</a>. Here we just highlight the main features.</p>
+<p>è¿æ¯ Kylin ä¸ä¸ä»£ v3.x ç第ä¸ä¸ªåå¸çæ¬ï¼ç¨äºæ©æé¢è§ï¼ä¸»è¦çåè½æ¯å®æ¶ ï¼Real-timeï¼ OLAPãå®æ´çæ¹å¨å表请åè§<a href="/docs/release_notes.html">release notes</a>ï¼è¿éæä¸äºä¸»è¦æ¹è¿å说æã</p>
-<h1 id="important-features">Important features</h1>
+<h1 id="section">éè¦æ°åè½</h1>
-<h3 id="kylin-3654---real-time-olap">KYLIN-3654 - Real-time OLAP</h3>
-<p>With the newly introduced Kylin real-time receiver and coordinator components, Kylin can implement a millisecond-level data preparation delay for streaming data from sources like Apache Kafka. This means since v3.0 on, Kylin can support sub-second level OLAP over historical batch data, near real-time streaming as well as real-time streaming. The user can use one OLAP platform to serve different scenarios. This solution has been deployed and verified in early adopters like eBay since 2018. For how to enable it, please refer to <a href="/docs30/tutorial/realtime_olap.html">this tutorial</a>.</p>
+<h3 id="kylin-3654----olap">KYLIN-3654 - å®æ¶ OLAP</h3>
+<p>éçå¼å
¥æ°ç real-time receiver å coordinator ç»ä»¶ï¼Kylin è½å¤å®ç°æ¯«ç§çº§å«çæ°æ®åå¤å»¶è¿ï¼æ°æ®æºæ¥èªæµå¼æ°æ®å¦ Apache Kafkaãè¿æå³çï¼ä» v3.0 å¼å§ï¼Kylin æ¢è½å¤æ¯æåå²æ¹éæ°æ®ç OLAPï¼ä¹æ¯æ对æµå¼æ°æ®çåå®æ¶ï¼Near real-timeï¼ä»¥åå®å
¨å®æ¶(real-time)åæãç¨æ·å¯ä»¥ä½¿ç¨ä¸ä¸ª OLAP å¹³å°æ¥æå¡ä¸åç使ç¨åºæ¯ãæ¤æ¹æ¡å·²ç»å¨æ©æç¨æ·å¦ eBay å¾å°é¨ç½²åéªè¯ãå
³äºå¦ä½ä½¿ç¨æ¤åè½ï¼è¯·åè<a href="/docs30/tutorial/realtime_olap.html">æ¤æ
ç¨</a>ã</p>
-<h3 id="kylin-3795---submit-spark-jobs-via-apache-livy">KYLIN-3795 - Submit Spark jobs via Apache Livy</h3>
-<p>This feature allows the administrator to configure Kylin to integrate with Apache Livy (incubating) for Spark job submissions. The Spark job is submitted to the Livy Server through Livyâs REST API, instead of starting the Spark Driver process in local, which facilitates the management and monitoring of the Spark resources, and also releases the pressure of the nodes where the Kylin job server is running.</p>
+<h3 id="kylin-3795----apache-livy--spark-">KYLIN-3795 - éè¿ Apache Livy é交 Spark ä»»å¡</h3>
+<p>è¿ä¸ªåè½å
许管çå为 Kylin é
ç½®ä½¿ç¨ Apache Livy (incubating) æ¥å®æä»»å¡çé交ãSpark ä½ä¸çæ交éè¿ Livy ç REST API æ¥æ交ï¼èæ éå¨æ¬å°å¯å¨ Spark Driver è¿ç¨ï¼ä»èæ¹ä¾¿å¯¹ Spark èµæºç管ççæ§ï¼åæ¶ä¹éä½å¯¹ Kylin ä»»å¡è¿ç¨æå¨èç¹çååã</p>
-<h3 id="kylin-3820---a-curator-based-job-scheduler">KYLIN-3820 - A curator-based job scheduler</h3>
-<p>A new job scheduler is added to automatically discover the Kylin nodes and do an automatic leader selection among them (only the leader will submit jobs). With this feature, you can easily deploy and scale out Kylin nodes without manually update the node address in <code class="highlighter-rouge">kylin.properties</code> and restart Kylin to take effective.</p>
+<h3 id="kylin-3820----curator-">KYLIN-3820 - åºäº Curator çä»»å¡èç¹åé
åæå¡åç°</h3>
+<p>æ°å¢ä¸ç§åºäºApache Zookeeper å Curatorä½ä¸è°åº¦å¨ï¼å¯ä»¥èªå¨åç° Kylin èç¹ï¼å¹¶èªå¨åé
ä¸ä¸ªèç¹æ¥è¿è¡ä»»å¡ç管ç以åæ
éæ¢å¤ãæäºè¿ä¸ªåè½åï¼ç®¡çåå¯ä»¥æ´å 容æå°é¨ç½²åæ©å± Kylin èç¹ï¼èä¸åéè¦å¨ <code class="highlighter-rouge">kylin.properties</code> ä¸é
ç½®æ¯ä¸ª Kylin èç¹çå°å并éå¯ Kylin 以使ä¹çæã</p>
-<h1 id="other-enhancements">Other enhancements</h1>
+<h1 id="section-1">å
¶å®æ¹è¿</h1>
-<h3 id="kylin-3716---fastthreadlocal-replaces-threadlocal">KYLIN-3716 - FastThreadLocal replaces ThreadLocal</h3>
-<p>Using FastThreadLocal instead of ThreadLocal can improve Kylinâs overall performance to some extent.</p>
+<h3 id="kylin-3716---fastthreadlocal--threadlocal">KYLIN-3716 - FastThreadLocal æ¿æ¢ ThreadLocal</h3>
+<p>ä½¿ç¨ Netty ä¸ç FastThreadLocal æ¿ä»£ JDK åçç ThreadLocalï¼å¯ä»¥ä¸å®ç¨åº¦ä¸æå Kylin å¨é«å¹¶åä¸çæ§è½ã</p>
<h3 id="kylin-3867---enable-jdbc-to-use-key-store--trust-store-for-https-connection">KYLIN-3867 - Enable JDBC to use key store &amp; trust store for https connection</h3>
-<p>By using HTTPS, the authentication information used by JDBC is protected, making Kylin more secure.</p>
+<p>éè¿ä½¿ç¨HTTPSï¼ä¿æ¤äºJDBC使ç¨ç身份éªè¯ä¿¡æ¯ï¼ä½¿å¾Kylinæ´å å®å
¨</p>
<h3 id="kylin-3905---enable-shrunken-dictionary-default">KYLIN-3905 - Enable shrunken dictionary default</h3>
-<p>By default, the shrunken dictionary is enabled, and the precise counting scene for high cardinal dimensions can significantly reduce the build time.</p>
+<p>é»è®¤å¼å¯ shrunken dictionaryï¼é对é«åºç»´è¿è¡ç²¾ç¡®å»éçåºæ¯ï¼å¯ä»¥æ¾èåå°æ建ç¨æ¶ã</p>
<h3 id="kylin-3839---storage-clean-up-after-the-refreshing-and-deleting-a-segment">KYLIN-3839 - Storage clean up after the refreshing and deleting a segment</h3>
-<p>Clear unnecessary data files in a timely manner</p>
+<p>æ´å åæ¶å°æ¸
é¤ä¸å¿
è¦çæ°æ®æ件</p>
-<p><strong>Download</strong></p>
+<p><strong>ä¸è½½</strong></p>
-<p>To download Apache Kylin v3.0.0-alpha source code or binary package, visit the <a href="http://kylin.apache.org/download">download</a> page.</p>
+<p>è¦ä¸è½½Apache Kylin æºä»£ç æäºè¿å¶å
ï¼è¯·è®¿é®<a href="/download">ä¸è½½é¡µé¢</a> page.</p>
-<p><strong>Upgrade</strong></p>
+<p><strong>å级</strong></p>
-<p>Follow the <a href="/docs/howto/howto_upgrade.html">upgrade guide</a>.</p>
+<p>åè<a href="/docs/howto/howto_upgrade.html">å级æå</a>.</p>
-<p><strong>Feedback</strong></p>
+<p><strong>åé¦</strong></p>
-<p>If you face issue or question, please send mail to Apache Kylin dev or user mailing list:Â dev@kylin.apache.org , user@kylin.apache.org; Before sending, please make sure you have subscribed the mailing list by dropping an email to dev-subscribe@kylin.apache.org or user-subscribe@kylin.apache.org.</p>
+<p>å¦ææ¨éå°é®é¢æçé®ï¼è¯·åéé®ä»¶è³ Apache Kylin dev æ user é®ä»¶å表ï¼dev@kylin.apache.orgï¼user@kylin.apache.org; å¨åéä¹åï¼è¯·ç¡®ä¿æ¨å·²éè¿åéçµåé®ä»¶è³ dev-subscribe@kylin.apache.org æ user-subscribe@kylin.apache.org 订é
äºé®ä»¶å表ã</p>
-<p><em>Great thanks to everyone who contributed!</em></p>
+<p><em>é常æè°¢ææè´¡ç®Apache Kylinçæå!</em></p>
</description>
<pubDate>Fri, 19 Apr 2019 13:00:00 -0700</pubDate>
- <link>http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</link>
- <guid isPermaLink="true">http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</guid>
+ <link>http://kylin.apache.org/cn/blog/2019/04/19/release-v3.0.0-alpha/</link>
+ <guid isPermaLink="true">http://kylin.apache.org/cn/blog/2019/04/19/release-v3.0.0-alpha/</guid>
<category>blog</category>
@@ -225,55 +298,55 @@ Security: (depend on your security setti
</item>
<item>
- <title>Apache Kylin v3.0.0-alpha åå¸</title>
- <description><p>è¿æ¥ Apache Kylin 社åºå¾é«å
´å°å®£å¸ï¼Apache Kylin v3.0.0-alpha æ£å¼åå¸ã</p>
+ <title>Apache Kylin v3.0.0-alpha Release Announcement</title>
+ <description><p>The Apache Kylin community is pleased to announce the release of Apache Kylin v3.0.0-alpha.</p>
-<p>Apache Kylin æ¯ä¸ä¸ªå¼æºçåå¸å¼åæå¼æï¼æ¨å¨ä¸ºæ大æ°æ®éæä¾ SQL æ¥å£åå¤ç»´åæï¼OLAPï¼çè½åã</p>
+<p>Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Big Data supporting extremely large datasets.</p>
-<p>è¿æ¯ Kylin ä¸ä¸ä»£ v3.x ç第ä¸ä¸ªåå¸çæ¬ï¼ç¨äºæ©æé¢è§ï¼ä¸»è¦çåè½æ¯å®æ¶ ï¼Real-timeï¼ OLAPãå®æ´çæ¹å¨å表请åè§<a href="/docs/release_notes.html">release notes</a>ï¼è¿éæä¸äºä¸»è¦æ¹è¿å说æã</p>
+<p>This is the first release of the new generation v3.x, the main feature introduced is the Real-time OLAP. All of the changes can be found in the <a href="/docs/release_notes.html">release notes</a>. Here we just highlight the main features.</p>
-<h1 id="section">éè¦æ°åè½</h1>
+<h1 id="important-features">Important features</h1>
-<h3 id="kylin-3654----olap">KYLIN-3654 - å®æ¶ OLAP</h3>
-<p>éçå¼å
¥æ°ç real-time receiver å coordinator ç»ä»¶ï¼Kylin è½å¤å®ç°æ¯«ç§çº§å«çæ°æ®åå¤å»¶è¿ï¼æ°æ®æºæ¥èªæµå¼æ°æ®å¦ Apache Kafkaãè¿æå³çï¼ä» v3.0 å¼å§ï¼Kylin æ¢è½å¤æ¯æåå²æ¹éæ°æ®ç OLAPï¼ä¹æ¯æ对æµå¼æ°æ®çåå®æ¶ï¼Near real-timeï¼ä»¥åå®å
¨å®æ¶(real-time)åæãç¨æ·å¯ä»¥ä½¿ç¨ä¸ä¸ª OLAP å¹³å°æ¥æå¡ä¸åç使ç¨åºæ¯ãæ¤æ¹æ¡å·²ç»å¨æ©æç¨æ·å¦ eBay å¾å°é¨ç½²åéªè¯ãå
³äºå¦ä½ä½¿ç¨æ¤åè½ï¼è¯·åè<a href="/docs30/tutorial/realtime_olap.html">æ¤æ
ç¨</a>ã</p>
+<h3 id="kylin-3654---real-time-olap">KYLIN-3654 - Real-time OLAP</h3>
+<p>With the newly introduced Kylin real-time receiver and coordinator components, Kylin can implement a millisecond-level data preparation delay for streaming data from sources like Apache Kafka. This means since v3.0 on, Kylin can support sub-second level OLAP over historical batch data, near real-time streaming as well as real-time streaming. The user can use one OLAP platform to serve different scenarios. This solution has been deployed and verified in early adopters like eBay since 2018. For how to enable it, please refer to <a href="/docs30/tutorial/realtime_olap.html">this tutorial</a>.</p>
-<h3 id="kylin-3795----apache-livy--spark-">KYLIN-3795 - éè¿ Apache Livy é交 Spark ä»»å¡</h3>
-<p>è¿ä¸ªåè½å
许管çå为 Kylin é
ç½®ä½¿ç¨ Apache Livy (incubating) æ¥å®æä»»å¡çé交ãSpark ä½ä¸çæ交éè¿ Livy ç REST API æ¥æ交ï¼èæ éå¨æ¬å°å¯å¨ Spark Driver è¿ç¨ï¼ä»èæ¹ä¾¿å¯¹ Spark èµæºç管ççæ§ï¼åæ¶ä¹éä½å¯¹ Kylin ä»»å¡è¿ç¨æå¨èç¹çååã</p>
+<h3 id="kylin-3795---submit-spark-jobs-via-apache-livy">KYLIN-3795 - Submit Spark jobs via Apache Livy</h3>
+<p>This feature allows the administrator to configure Kylin to integrate with Apache Livy (incubating) for Spark job submissions. The Spark job is submitted to the Livy Server through Livyâs REST API, instead of starting the Spark Driver process in local, which facilitates the management and monitoring of the Spark resources, and also releases the pressure of the nodes where the Kylin job server is running.</p>
-<h3 id="kylin-3820----curator-">KYLIN-3820 - åºäº Curator çä»»å¡èç¹åé
åæå¡åç°</h3>
-<p>æ°å¢ä¸ç§åºäºApache Zookeeper å Curatorä½ä¸è°åº¦å¨ï¼å¯ä»¥èªå¨åç° Kylin èç¹ï¼å¹¶èªå¨åé
ä¸ä¸ªèç¹æ¥è¿è¡ä»»å¡ç管ç以åæ
éæ¢å¤ãæäºè¿ä¸ªåè½åï¼ç®¡çåå¯ä»¥æ´å 容æå°é¨ç½²åæ©å± Kylin èç¹ï¼èä¸åéè¦å¨ <code class="highlighter-rouge">kylin.properties</code> ä¸é
ç½®æ¯ä¸ª Kylin èç¹çå°å并éå¯ Kylin 以使ä¹çæã</p>
+<h3 id="kylin-3820---a-curator-based-job-scheduler">KYLIN-3820 - A curator-based job scheduler</h3>
+<p>A new job scheduler is added to automatically discover the Kylin nodes and do an automatic leader selection among them (only the leader will submit jobs). With this feature, you can easily deploy and scale out Kylin nodes without manually update the node address in <code class="highlighter-rouge">kylin.properties</code> and restart Kylin to take effective.</p>
-<h1 id="section-1">å
¶å®æ¹è¿</h1>
+<h1 id="other-enhancements">Other enhancements</h1>
-<h3 id="kylin-3716---fastthreadlocal--threadlocal">KYLIN-3716 - FastThreadLocal æ¿æ¢ ThreadLocal</h3>
-<p>ä½¿ç¨ Netty ä¸ç FastThreadLocal æ¿ä»£ JDK åçç ThreadLocalï¼å¯ä»¥ä¸å®ç¨åº¦ä¸æå Kylin å¨é«å¹¶åä¸çæ§è½ã</p>
+<h3 id="kylin-3716---fastthreadlocal-replaces-threadlocal">KYLIN-3716 - FastThreadLocal replaces ThreadLocal</h3>
+<p>Using FastThreadLocal instead of ThreadLocal can improve Kylinâs overall performance to some extent.</p>
<h3 id="kylin-3867---enable-jdbc-to-use-key-store--trust-store-for-https-connection">KYLIN-3867 - Enable JDBC to use key store &amp; trust store for https connection</h3>
-<p>éè¿ä½¿ç¨HTTPSï¼ä¿æ¤äºJDBC使ç¨ç身份éªè¯ä¿¡æ¯ï¼ä½¿å¾Kylinæ´å å®å
¨</p>
+<p>By using HTTPS, the authentication information used by JDBC is protected, making Kylin more secure.</p>
<h3 id="kylin-3905---enable-shrunken-dictionary-default">KYLIN-3905 - Enable shrunken dictionary default</h3>
-<p>é»è®¤å¼å¯ shrunken dictionaryï¼é对é«åºç»´è¿è¡ç²¾ç¡®å»éçåºæ¯ï¼å¯ä»¥æ¾èåå°æ建ç¨æ¶ã</p>
+<p>By default, the shrunken dictionary is enabled, and the precise counting scene for high cardinal dimensions can significantly reduce the build time.</p>
<h3 id="kylin-3839---storage-clean-up-after-the-refreshing-and-deleting-a-segment">KYLIN-3839 - Storage clean up after the refreshing and deleting a segment</h3>
-<p>æ´å åæ¶å°æ¸
é¤ä¸å¿
è¦çæ°æ®æ件</p>
+<p>Clear unnecessary data files in a timely manner</p>
-<p><strong>ä¸è½½</strong></p>
+<p><strong>Download</strong></p>
-<p>è¦ä¸è½½Apache Kylin æºä»£ç æäºè¿å¶å
ï¼è¯·è®¿é®<a href="/download">ä¸è½½é¡µé¢</a> page.</p>
+<p>To download Apache Kylin v3.0.0-alpha source code or binary package, visit the <a href="http://kylin.apache.org/download">download</a> page.</p>
-<p><strong>å级</strong></p>
+<p><strong>Upgrade</strong></p>
-<p>åè<a href="/docs/howto/howto_upgrade.html">å级æå</a>.</p>
+<p>Follow the <a href="/docs/howto/howto_upgrade.html">upgrade guide</a>.</p>
-<p><strong>åé¦</strong></p>
+<p><strong>Feedback</strong></p>
-<p>å¦ææ¨éå°é®é¢æçé®ï¼è¯·åéé®ä»¶è³ Apache Kylin dev æ user é®ä»¶å表ï¼dev@kylin.apache.orgï¼user@kylin.apache.org; å¨åéä¹åï¼è¯·ç¡®ä¿æ¨å·²éè¿åéçµåé®ä»¶è³ dev-subscribe@kylin.apache.org æ user-subscribe@kylin.apache.org 订é
äºé®ä»¶å表ã</p>
+<p>If you face issue or question, please send mail to Apache Kylin dev or user mailing list:Â dev@kylin.apache.org , user@kylin.apache.org; Before sending, please make sure you have subscribed the mailing list by dropping an email to dev-subscribe@kylin.apache.org or user-subscribe@kylin.apache.org.</p>
-<p><em>é常æè°¢ææè´¡ç®Apache Kylinçæå!</em></p>
+<p><em>Great thanks to everyone who contributed!</em></p>
</description>
<pubDate>Fri, 19 Apr 2019 13:00:00 -0700</pubDate>
- <link>http://kylin.apache.org/cn/blog/2019/04/19/release-v3.0.0-alpha/</link>
- <guid isPermaLink="true">http://kylin.apache.org/cn/blog/2019/04/19/release-v3.0.0-alpha/</guid>
+ <link>http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</link>
+ <guid isPermaLink="true">http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</guid>
<category>blog</category>
@@ -1107,70 +1180,6 @@ Graphic 10 Process of Querying Cube</
<category>blog</category>
-
- </item>
-
- <item>
- <title>Apache Kylin v2.5.0 æ£å¼åå¸</title>
- <description><p>è¿æ¥Apache Kylin 社åºå¾é«å
´å°å®£å¸ï¼Apache Kylin 2.5.0 æ£å¼åå¸ã</p>
-
-<p>Apache Kylin æ¯ä¸ä¸ªå¼æºçåå¸å¼åæå¼æï¼æ¨å¨ä¸ºæ大æ°æ®éæä¾ SQL æ¥å£åå¤ç»´åæï¼OLAPï¼çè½åã</p>
-
-<p>è¿æ¯ç»§2.4.0 åçä¸ä¸ªæ°åè½çæ¬ã该çæ¬å¼å
¥äºå¾å¤æä»·å¼çæ¹è¿ï¼å®æ´çæ¹å¨å表请åè§<a href="https://kylin.apache.org/docs/release_notes.html">release notes</a>ï¼è¿éæä¸äºä¸»è¦æ¹è¿å说æï¼</p>
-
-<h3 id="all-in-spark--cubing-">All-in-Spark ç Cubing å¼æ</h3>
-<p>Kylin ç Spark å¼æå°ä½¿ç¨ Spark è¿è¡ cube 计ç®ä¸çææåå¸å¼ä½ä¸ï¼å
æ¬è·åå个维度çä¸åå¼ï¼å° cuboid æ件转æ¢ä¸º HBase HFileï¼å并 segmentï¼å并è¯å
¸çãé»è®¤ç Spark é
ç½®ä¹ç»è¿ä¼åï¼ä½¿å¾ç¨æ·å¯ä»¥è·å¾å¼ç®±å³ç¨çä½éªãç¸å
³å¼åä»»å¡æ¯ KYLIN-3427, KYLIN-3441, KYLIN-3442.</p>
-
-<p>Spark ä»»å¡ç®¡çä¹æææ¹è¿ï¼ä¸æ¦ Spark ä»»å¡å¼å§è¿è¡ï¼æ¨å°±å¯ä»¥å¨Webæ§å¶å°ä¸è·å¾ä½ä¸é¾æ¥ï¼å¦ææ¨ä¸¢å¼è¯¥ä½ä¸ï¼Kylin å°ç«å»ç»æ¢ Spark ä½ä¸ä»¥åæ¶éæ¾èµæºï¼å¦æéæ°å¯å¨ Kylinï¼å®å¯ä»¥ä»ä¸ä¸ä¸ªä½ä¸æ¢å¤ï¼èä¸æ¯éæ°æ交æ°ä½ä¸.</p>
-
-<h3 id="mysql--kylin-">MySQL å Kylin å
æ°æ®çåå¨</h3>
-<p>å¨è¿å»ï¼HBase æ¯ Kylin å
æ°æ®åå¨çå¯ä¸éæ©ã å¨æäºæ
åµä¸ HBaseä¸éç¨ï¼ä¾å¦ä½¿ç¨å¤ä¸ª HBase é群æ¥ä¸º Kylin æä¾è·¨åºåçé«å¯ç¨ï¼è¿éå¤å¶ç HBase é群æ¯åªè¯»çï¼æ以ä¸è½åå
æ°æ®åå¨ãç°å¨æ们å¼å
¥äº MySQL Metastore 以满足è¿ç§éæ±ãæ¤åè½ç°å¨å¤äºæµè¯é¶æ®µãæ´å¤å
容åè§ KYLIN-3488ã</p>
-
-<h3 id="hybrid-model-">Hybrid model å¾å½¢çé¢</h3>
-<p>Hybrid æ¯ä¸ç§ç¨äºç»è£
å¤ä¸ª cube çé«çº§æ¨¡åã å®å¯ç¨äºæ»¡è¶³ cube ç schema è¦åçæ¹åçæ
åµãè¿ä¸ªåè½è¿å»æ²¡æå¾å½¢çé¢ï¼å æ¤åªæä¸å°é¨åç¨æ·ç¥éå®ãç°å¨æä»¬å¨ Web çé¢ä¸å¼å¯äºå®ï¼ä»¥ä¾¿æ´å¤ç¨æ·å¯ä»¥å°è¯ã</p>
-
-<h3 id="cube-planner">é»è®¤å¼å¯ Cube planner</h3>
-<p>Cube planner å¯ä»¥æ大å°ä¼å cube ç»æï¼åå°æ建ç cuboid æ°éï¼ä»èèç计ç®/åå¨èµæºå¹¶æé«æ¥è¯¢æ§è½ãå®æ¯å¨v2.3ä¸å¼å
¥çï¼ä½é»è®¤æ
åµä¸æ²¡æå¼å¯ã为äºè®©æ´å¤ç¨æ·çå°å¹¶å°è¯å®ï¼æ们é»è®¤å¨v2.5ä¸å¯ç¨å®ã ç®æ³å°å¨ç¬¬ä¸æ¬¡æ建 segment çæ¶åï¼æ ¹æ®æ°æ®ç»è®¡èªå¨ä¼å cuboid éå.</p>
-
-<h3 id="segment-">æ¹è¿ç Segment åªæ</h3>
-<p>Segmentï¼ååºï¼ä¿®åªå¯ä»¥ææå°åå°ç£çåç½ç»I / Oï¼å æ¤å¤§å¤§æé«äºæ¥è¯¢æ§è½ã è¿å»ï¼Kylin åªæååºå (partition date column) çå¼è¿è¡ segment çä¿®åªã å¦ææ¥è¯¢ä¸æ²¡æå°ååºåä½ä¸ºè¿æ»¤æ¡ä»¶ï¼é£ä¹ä¿®åªå°ä¸èµ·ä½ç¨ï¼ä¼æ«æææsegmentã.<br />
-ç°å¨ä»v2.5å¼å§ï¼Kylin å°å¨ segment 级å«è®°å½æ¯ä¸ªç»´åº¦çæå°/æ大å¼ã å¨æ«æ segment ä¹åï¼ä¼å°æ¥è¯¢çæ¡ä»¶ä¸æå°/æ大索å¼è¿è¡æ¯è¾ã å¦æä¸å¹é
ï¼å°è·³è¿è¯¥ segmentã æ£æ¥KYLIN-3370äºè§£æ´å¤ä¿¡æ¯ã</p>
-
-<h3 id="yarn-">å¨ YARN ä¸å并åå
¸</h3>
-<p>å½ segment å并æ¶ï¼å®ä»¬çè¯å
¸ä¹éè¦å并ãå¨è¿å»ï¼åå
¸å并åçå¨ Kylin ç JVM ä¸ï¼è¿éè¦ä½¿ç¨å¤§éçæ¬å°å
åå CPU èµæºã å¨æ端æ
åµä¸ï¼å¦ææå 个并åä½ä¸ï¼ï¼å¯è½ä¼å¯¼è´ Kylin è¿ç¨å´©æºã å æ¤ï¼ä¸äºç¨æ·ä¸å¾ä¸ä¸º Kylin ä»»å¡èç¹åé
æ´å¤å
åï¼æè¿è¡å¤ä¸ªä»»å¡èç¹ä»¥å¹³è¡¡å·¥ä½è´è½½ã<br />
-ç°å¨ä»v2.5å¼å§ï¼Kylin å°æè¿é¡¹ä»»å¡æäº¤ç» Hadoop MapReduce å Sparkï¼è¿æ ·å°±å¯ä»¥è§£å³è¿ä¸ªç¶é¢é®é¢ã æ¥çKYLIN-3471äºè§£æ´å¤ä¿¡æ¯.</p>
-
-<h3 id="cube-">æ¹è¿ä½¿ç¨å
¨å±åå
¸ç cube æ建æ§è½</h3>
-<p>å
¨å±åå
¸ (Global Dictionary) æ¯ bitmap 精确å»é计æ°çå¿
è¦æ¡ä»¶ãå¦æå»éåå
·æé常é«çåºæ°ï¼å GD å¯è½é常大ãå¨ cube æ建é¶æ®µï¼Kylin éè¦éè¿ GD å°éæ´æ°å¼è½¬æ¢ä¸ºæ´æ°ã尽管 GD 已被åæå¤ä¸ªåçï¼å¯ä»¥åå¼å è½½å°å
åï¼ä½æ¯ç±äºå»éåçå¼æ¯ä¹±åºçãKylin éè¦åå¤è½½å
¥åè½½åº(swap in/out)åçï¼è¿ä¼å¯¼è´æ建任å¡é常ç¼æ
¢ã<br />
-该å¢å¼ºåè½å¼å
¥äºä¸ä¸ªæ°æ¥éª¤ï¼ä¸ºæ¯ä¸ªæ°æ®åä»å
¨å±åå
¸ä¸æ建ä¸ä¸ªç¼©å°çåå
¸ã éåæ¯ä¸ªä»»å¡åªéè¦å 载缩å°çåå
¸ï¼ä»èé¿å
é¢ç¹çè½½å
¥åè½½åºãæ§è½å¯ä»¥æ¯ä»¥åå¿«3åãæ¥ç KYLIN-3491 äºè§£æ´å¤ä¿¡æ¯.</p>
-
-<h3 id="topn-count-distinct--cube-">æ¹è¿å« TOPN, COUNT DISTINCT ç cube 大å°ç估计</h3>
-<p>Cube ç大å°å¨æ建æ¶æ¯é¢å
估计çï¼å¹¶è¢«åç»å 个æ¥éª¤ä½¿ç¨ï¼ä¾å¦å³å® MR / Spark ä½ä¸çååºæ°ï¼è®¡ç® HBase region åå²çãå®çåç¡®ä¸å¦ä¼å¯¹æ建æ§è½äº§çå¾å¤§å½±åã å½åå¨ COUNT DISTINCTï¼TOPN ç度éæ¶åï¼å 为å®ä»¬ç大å°æ¯çµæ´»çï¼å æ¤ä¼°è®¡å¼å¯è½è·çå®å¼æå¾å¤§åå·®ã å¨è¿å»ï¼ç¨æ·éè¦è°æ´è¥å¹²ä¸ªåæ°ä»¥ä½¿å°ºå¯¸ä¼°è®¡æ´æ¥è¿å®é
尺寸ï¼è¿å¯¹æ®éç¨æ·æç¹å°é¾ã<br />
-ç°å¨ï¼Kylin å°æ ¹æ®æ¶éçç»è®¡ä¿¡æ¯èªå¨è°æ´å¤§å°ä¼°è®¡ãè¿å¯ä»¥ä½¿ä¼°è®¡å¼ä¸å®é
大å°æ´æ¥è¿ãæ¥ç KYLIN-3453 äºè§£æ´å¤ä¿¡æ¯ã</p>
-
-<h3 id="hadoop-30hbase-20">æ¯æHadoop 3.0/HBase 2.0</h3>
-<p>Hadoop 3å HBase 2å¼å§è¢«è®¸å¤ç¨æ·éç¨ãç°å¨ Kylin æä¾ä½¿ç¨æ°ç Hadoop å HBase API ç¼è¯çæ°äºè¿å¶å
ãæ们已ç»å¨ Hortonworks HDP 3.0 å Cloudera CDH 6.0 ä¸è¿è¡äºæµè¯</p>
-
-<p><strong>ä¸è½½</strong></p>
-
-<p>è¦ä¸è½½Apache Kylin v2.5.0æºä»£ç æäºè¿å¶å
ï¼è¯·è®¿é®<a href="http://kylin.apache.org/download">ä¸è½½é¡µé¢</a> .</p>
-
-<p><strong>å级</strong></p>
-
-<p>åè<a href="/docs/howto/howto_upgrade.html">å级æå</a>.</p>
-
-<p><strong>åé¦</strong></p>
-
-<p>å¦ææ¨éå°é®é¢æçé®ï¼è¯·åéé®ä»¶è³ Apache Kylin dev æ user é®ä»¶å表ï¼dev@kylin.apache.orgï¼user@kylin.apache.org; å¨åéä¹åï¼è¯·ç¡®ä¿æ¨å·²éè¿åéçµåé®ä»¶è³ dev-subscribe@kylin.apache.org æ user-subscribe@kylin.apache.org订é
äºé®ä»¶å表ã</p>
-
-<p><em>é常æè°¢ææè´¡ç®Apache Kylinçæå!</em></p>
-</description>
- <pubDate>Thu, 20 Sep 2018 13:00:00 -0700</pubDate>
- <link>http://kylin.apache.org/cn/blog/2018/09/20/release-v2.5.0/</link>
- <guid isPermaLink="true">http://kylin.apache.org/cn/blog/2018/09/20/release-v2.5.0/</guid>
-
-
- <category>blog</category>
</item>