You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kylin.apache.org by li...@apache.org on 2019/08/01 14:04:53 UTC

svn commit: r1864154 [17/17] - in /kylin/site: ./ blog/ blog/2019/07/30/ blog/2019/07/30/detailed-analysis-of-refine-query-cache/ cn/docs30/ cn/docs30/gettingstarted/ cn/docs30/howto/ cn/docs30/install/ cn/docs30/tutorial/ docs30/ docs30/gettingstarted...

Modified: kylin/site/feed.xml
URL: http://svn.apache.org/viewvc/kylin/site/feed.xml?rev=1864154&r1=1864153&r2=1864154&view=diff
==============================================================================
--- kylin/site/feed.xml (original)
+++ kylin/site/feed.xml Thu Aug  1 14:04:52 2019
@@ -19,11 +19,281 @@
     <description>Apache Kylin Home</description>
     <link>http://kylin.apache.org/</link>
     <atom:link href="http://kylin.apache.org/feed.xml" rel="self" type="application/rss+xml"/>
-    <pubDate>Thu, 01 Aug 2019 04:10:17 -0700</pubDate>
-    <lastBuildDate>Thu, 01 Aug 2019 04:10:17 -0700</lastBuildDate>
+    <pubDate>Thu, 01 Aug 2019 06:52:56 -0700</pubDate>
+    <lastBuildDate>Thu, 01 Aug 2019 06:52:56 -0700</lastBuildDate>
     <generator>Jekyll v2.5.3</generator>
     
       <item>
+        <title>Detailed Analysis of refine query cache</title>
+        <description>&lt;hr /&gt;
+
+&lt;h2 id=&quot;part-i-basic-introduction&quot;&gt;Part-I Basic Introduction&lt;/h2&gt;
+
+&lt;h3 id=&quot;backgroud&quot;&gt;Backgroud&lt;/h3&gt;
+&lt;p&gt;In the past, query cache are not efficiently used in Kylin due to two aspects: &lt;strong&gt;coarse-grained cache expiration strategy&lt;/strong&gt; and &lt;strong&gt;lack of external cache&lt;/strong&gt;. Because of the aggressive cache expiration strategy, useful caches are often cleaned up unnecessarily. Because query caches are stored in local servers, they cannot be shared between servers. And because of the size limitation of local cache, not all useful query results can be cached.&lt;/p&gt;
+
+&lt;p&gt;To deal with these shortcomings, we change the query cache expiration strategy by signature checking and introduce the memcached as Kylin’s distributed cache so that Kylin servers are able to share cache between servers. And it’s easy to add memcached servers to scale out distributed cache.&lt;/p&gt;
+
+&lt;p&gt;These features is proposed and developed by eBay Kylin team. Thanks so much for their contribution.&lt;/p&gt;
+
+&lt;h3 id=&quot;related-jira&quot;&gt;Related JIRA&lt;/h3&gt;
+
+&lt;ul&gt;
+  &lt;li&gt;&lt;a href=&quot;https://issues.apache.org/jira/browse/KYLIN-2895&quot;&gt;KYLIN-2895 Refine Query Cache&lt;/a&gt;
+    &lt;ul&gt;
+      &lt;li&gt;&lt;a href=&quot;https://issues.apache.org/jira/browse/KYLIN-2899&quot;&gt;KYLIN-2899 Introduce segment level query cache&lt;/a&gt;&lt;/li&gt;
+      &lt;li&gt;&lt;a href=&quot;https://issues.apache.org/jira/browse/KYLIN-2898&quot;&gt;KYLIN-2898 Introduce memcached as a distributed cache for queries&lt;/a&gt;&lt;/li&gt;
+      &lt;li&gt;&lt;a href=&quot;https://issues.apache.org/jira/browse/KYLIN-2894&quot;&gt;KYLIN-2894 Change the query cache expiration strategy by signature checking&lt;/a&gt;&lt;/li&gt;
+      &lt;li&gt;&lt;a href=&quot;https://issues.apache.org/jira/browse/KYLIN-2897&quot;&gt;KYLIN-2897 Improve the query execution for a set of duplicate queries in a short period&lt;/a&gt;&lt;/li&gt;
+      &lt;li&gt;&lt;a href=&quot;https://issues.apache.org/jira/browse/KYLIN-2896&quot;&gt;KYLIN-2896 Refine query exception cache&lt;/a&gt;&lt;/li&gt;
+    &lt;/ul&gt;
+  &lt;/li&gt;
+&lt;/ul&gt;
+
+&lt;hr /&gt;
+
+&lt;h2 id=&quot;part-ii-deep-dive&quot;&gt;Part-II Deep Dive&lt;/h2&gt;
+
+&lt;ul&gt;
+  &lt;li&gt;Introduce memcached as a Distributed Query Cache&lt;/li&gt;
+  &lt;li&gt;Segment Level Cache&lt;/li&gt;
+  &lt;li&gt;Query Cache Expiration Strategy by Signature Checking&lt;/li&gt;
+  &lt;li&gt;Other Enhancement&lt;/li&gt;
+&lt;/ul&gt;
+
+&lt;h3 id=&quot;introduce-memcached-as-a-distributed-query-cache&quot;&gt;Introduce memcached as a Distributed Query Cache&lt;/h3&gt;
+
+&lt;p&gt;&lt;strong&gt;Memcached&lt;/strong&gt; is a Free and open source, high-performance, distributed memory object caching system. It is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering. It is simple yet powerful. Its simple design promotes quick deployment, ease of development, and solves many problems facing large data caches. Its API is available for most popular languages.&lt;/p&gt;
+
+&lt;p&gt;By KYLIN-2898, Kylin use &lt;strong&gt;Memcached&lt;/strong&gt; as distributed cache service, and use &lt;strong&gt;EhCache&lt;/strong&gt; as local cache service. When &lt;code class=&quot;highlighter-rouge&quot;&gt;RemoteLocalFailOverCacheManager&lt;/code&gt; is configured in &lt;code class=&quot;highlighter-rouge&quot;&gt;applicationContext.xml&lt;/code&gt;, for each cache put/get action, Kylin will first check if remote cache service is available, only if remote cache service is unavailable, local cache service will be used.&lt;/p&gt;
+
+&lt;p&gt;Firstly, multi query server can share query cache. For each kylin server, less jvm memory will be occupied which help to reduce GC pressure. Secondly, since memcached is centralized so duplicated cache entry will avoid in serval Kylin process. Thirdly, memcached has larger size and easy to scale out, this will help to reduce the chance which useful cache entry have to be dropped due to limited memory capacity.&lt;/p&gt;
+
+&lt;p&gt;To handle node failure and to scale out memcached cluster, author has introduced a consistent hash strategy to smoothly solve such problem. Ketama is an implementation of a consistent hashing algorithm, meaning you can add or remove servers from the memcached pool without causing a complete remap of all keys. Detail could be checked at &lt;a href=&quot;https://www.last.fm/user/RJ/journal/2007/04/10/rz_libketama_-_a_consistent_hashing_algo_for_memcache_clients&quot;&gt;Ketama consistent hash strategy&lt;/a&gt;.&lt;/p&gt;
+
+&lt;p&gt;&lt;img src=&quot;/images/blog/refine-query-cache/consistent-hashing.png&quot; alt=&quot;consistent hashing&quot; /&gt;&lt;/p&gt;
+
+&lt;h3 id=&quot;segment-level-cache&quot;&gt;Segment level Cache&lt;/h3&gt;
+
+&lt;p&gt;Currently Kylin use sql as the cache key, when sql comes, if result exists in the cache, it will directly returned the cached result and don’t need to query hbase. When there is new segment build or existing segment refresh, all related cache result need to be evicted. For some frequently build cube such as streaming cube(NRT Streaming or Real-time OLAP), the cache miss will increase dramatically, that may decrease the query performance.&lt;/p&gt;
+
+&lt;p&gt;Since for Kylin cube, most historical segments are immutable, the same query against historical segments should be always same, don’t need to be evicted for new segment building. So we decide to implement the segment level cache, it is a complement of the existing front-end cache, the idea is similar as the level1/level2 cache in operating system.&lt;/p&gt;
+
+&lt;p&gt;&lt;img src=&quot;/images/blog/refine-query-cache/l1-l2-cache.png&quot; alt=&quot;l1-l2-cache&quot; /&gt;&lt;/p&gt;
+
+&lt;h3 id=&quot;query-cache-expiration-strategy-by-signature-checking&quot;&gt;Query Cache Expiration Strategy by Signature Checking&lt;/h3&gt;
+
+&lt;p&gt;Currently, to invalid query cache, &lt;code class=&quot;highlighter-rouge&quot;&gt;CacheService&lt;/code&gt; will either invoke &lt;code class=&quot;highlighter-rouge&quot;&gt;cleanDataCache&lt;/code&gt; or &lt;code class=&quot;highlighter-rouge&quot;&gt;cleanAllDataCache&lt;/code&gt;. Both methods will clear all of the query cache , which is very inefficient and unnecessary. In production environment, there’s around hundreds of cubing jobs per day, which means the query cache will be cleared very several minutes. Then we introduced a signature to upgrade cache invalidation strategy.&lt;/p&gt;
+
+&lt;p&gt;The basic idea is as follows:&lt;br /&gt;
+When put SQLResponse into cache, we add signature for each SQLResponse. To calculate signature for SQLResponse, we choose the cube last build time and its segments to as input of &lt;code class=&quot;highlighter-rouge&quot;&gt;SignatureCalculator&lt;/code&gt;.&lt;br /&gt;
+When fetch &lt;code class=&quot;highlighter-rouge&quot;&gt;SQLResponse&lt;/code&gt; for cache, first check whether the signature is consistent. If not, this cached value is overdue and will be invalidate.&lt;/p&gt;
+
+&lt;p&gt;As for the calculation of signature is show as follows:&lt;br /&gt;
+1. &lt;code class=&quot;highlighter-rouge&quot;&gt;toString&lt;/code&gt; of &lt;code class=&quot;highlighter-rouge&quot;&gt;ComponentSignature&lt;/code&gt; will concatenate member varible into a large String; if a &lt;code class=&quot;highlighter-rouge&quot;&gt;ComponentSignature&lt;/code&gt; has other &lt;code class=&quot;highlighter-rouge&quot;&gt;ComponentSignature&lt;/code&gt; as member, toString will be calculated recursively&lt;br /&gt;
+2. return value of &lt;code class=&quot;highlighter-rouge&quot;&gt;toString&lt;/code&gt; will be input of &lt;code class=&quot;highlighter-rouge&quot;&gt;SignatureCalculator&lt;/code&gt;,&lt;br /&gt;
+&lt;code class=&quot;highlighter-rouge&quot;&gt;SignatureCalculator&lt;/code&gt; encode string using MD5 as identifer of signature of query cache&lt;/p&gt;
+
+&lt;p&gt;&lt;img src=&quot;/images/blog/refine-query-cache/cache-signature.png&quot; alt=&quot;cache-signature&quot; /&gt;&lt;/p&gt;
+
+&lt;h3 id=&quot;other-enhancement&quot;&gt;Other Enhancement&lt;/h3&gt;
+
+&lt;h4 id=&quot;improve-the-query-execution-for-a-set-of-duplicate-queries-in-a-short-period&quot;&gt;Improve the query execution for a set of duplicate queries in a short period&lt;/h4&gt;
+
+&lt;p&gt;If same query enter Kylin at the same time by different client, for each query they can not find query cache so they must be calculated respectively. And even wrose, if these query are complex, they usually cost a long duration so Kylin have less chance to utilize cache query; and them cost large computation resources that will make query server has poor performance has harm to HBase cluster.&lt;/p&gt;
+
+&lt;p&gt;To reduce the impact of duplicated and complex query, it may be a good idea to block query which came later, wait to first one return result as far as possible. This lazy strategy is especially useful if you have duplicated complex query came in same time. To enbale it, you should set &lt;code class=&quot;highlighter-rouge&quot;&gt;kylin.query.lazy-query-enabled&lt;/code&gt; to &lt;code class=&quot;highlighter-rouge&quot;&gt;true&lt;/code&gt;. Optionlly, you may set &lt;code class=&quot;highlighter-rouge&quot;&gt;kylin.query.lazy-query-waiting-timeout-milliseconds&lt;/code&gt; to what you think later duplicated query wait duration to meet your situation.&lt;/p&gt;
+
+&lt;h4 id=&quot;remove-exception-cache&quot;&gt;Remove exception cache&lt;/h4&gt;
+&lt;p&gt;Formerly, query cache has been divided into two part, one part for storing success query result, another for failed query result, and they are invalidated respectively. It looks like not a good classification criteria because it is not fine-grained enough. After query cache signature was introduced, we have no reason to take them apart, so exception cache was removed.&lt;/p&gt;
+
+&lt;hr /&gt;
+
+&lt;h2 id=&quot;part-iii-how-to-use&quot;&gt;Part-III How to Use&lt;/h2&gt;
+
+&lt;p&gt;To get prepared, you need to install memcached, you may refer to https://github.com/memcached/memcached/wiki/Install. Then you should modify &lt;code class=&quot;highlighter-rouge&quot;&gt;kylin.properties&lt;/code&gt; and &lt;code class=&quot;highlighter-rouge&quot;&gt;applicationContext.xml&lt;/code&gt;.&lt;/p&gt;
+
+&lt;ul&gt;
+  &lt;li&gt;kylin.properties&lt;/li&gt;
+&lt;/ul&gt;
+
+&lt;div class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-groff&quot; data-lang=&quot;groff&quot;&gt;kylin.cache.memcached.hosts=10.1.2.42:11211
+kylin.query.cache-signature-enabled=true
+kylin.query.lazy-query-enabled=true
+kylin.metrics.memcached.enabled=true
+kylin.query.segment-cache-enabled=true&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
+
+&lt;ul&gt;
+  &lt;li&gt;applicationContext.xml&lt;/li&gt;
+&lt;/ul&gt;
+
+&lt;div class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-groff&quot; data-lang=&quot;groff&quot;&gt;&amp;lt;cache:annotation-driven/&amp;gt;
+
+&amp;lt;bean id=&quot;ehcache&quot; class=&quot;org.springframework.cache.ehcache.EhCacheManagerFactoryBean&quot;
+      p:configLocation=&quot;classpath:ehcache-test.xml&quot; p:shared=&quot;true&quot;/&amp;gt;
+
+&amp;lt;bean id=&quot;remoteCacheManager&quot; class=&quot;org.apache.kylin.cache.cachemanager.MemcachedCacheManager&quot;/&amp;gt;
+&amp;lt;bean id=&quot;localCacheManager&quot; class=&quot;org.apache.kylin.cache.cachemanager.InstrumentedEhCacheCacheManager&quot;
+      p:cacheManager-ref=&quot;ehcache&quot;/&amp;gt;
+&amp;lt;bean id=&quot;cacheManager&quot; class=&quot;org.apache.kylin.cache.cachemanager.RemoteLocalFailOverCacheManager&quot;/&amp;gt;
+
+&amp;lt;bean id=&quot;memcachedCacheConfig&quot; class=&quot;org.apache.kylin.cache.memcached.MemcachedCacheConfig&quot;&amp;gt;
+    &amp;lt;property name=&quot;timeout&quot; value=&quot;500&quot;/&amp;gt;
+    &amp;lt;property name=&quot;hosts&quot; value=&quot;${kylin.cache.memcached.hosts}&quot;/&amp;gt;
+&amp;lt;/bean&amp;gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
+
+&lt;h3 id=&quot;configuration-for-query-cache&quot;&gt;Configuration for query cache&lt;/h3&gt;
+
+&lt;h4 id=&quot;general-part&quot;&gt;General part&lt;/h4&gt;
+
+&lt;table&gt;
+  &lt;thead&gt;
+    &lt;tr&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf Key&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf value&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Explanation&lt;/th&gt;
+    &lt;/tr&gt;
+  &lt;/thead&gt;
+  &lt;tbody&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;kylin.query.cache-enabled&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;boolean, default true&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;whether to enable query cache&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;kylin.query.cache-threshold-duration&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;long, in milliseconds, default is 2000&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;query duration threshold&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;kylin.query.cache-threshold-scan-count&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;long, default is 10240&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;query scan row count threshold&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;kylin.query.cache-threshold-scan-bytes&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;long, default is 1024 * 1024 (1MB)&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;query scan byte threshold&lt;/td&gt;
+    &lt;/tr&gt;
+  &lt;/tbody&gt;
+&lt;/table&gt;
+
+&lt;h4 id=&quot;memcached-part&quot;&gt;Memcached part&lt;/h4&gt;
+
+&lt;table&gt;
+  &lt;thead&gt;
+    &lt;tr&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf Key&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf value&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Explanation&lt;/th&gt;
+    &lt;/tr&gt;
+  &lt;/thead&gt;
+  &lt;tbody&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;kylin.cache.memcached.hosts&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;host1:port1,host2:port2&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;host list of memcached host&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;kylin.query.segment-cache-enabled&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;default false&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;wether to enable&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;kylin.query.segment-cache-timeout&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;default 2000&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;timeout of memcached&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;kylin.query.segment-cache-max-size&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;200 (MB)&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;max size put into memcached&lt;/td&gt;
+    &lt;/tr&gt;
+  &lt;/tbody&gt;
+&lt;/table&gt;
+
+&lt;h4 id=&quot;cache-signature-part&quot;&gt;Cache signature part&lt;/h4&gt;
+
+&lt;table&gt;
+  &lt;thead&gt;
+    &lt;tr&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf Key&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf value&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Explanation&lt;/th&gt;
+    &lt;/tr&gt;
+  &lt;/thead&gt;
+  &lt;tbody&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;kylin.query.cache-signature-enabled&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;default false&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;whether to use signature for query cache&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;kylin.query.signature-class&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;default is org.apache.kylin.rest.signature.FactTableRealizationSetCalculator&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;use which class to calculate signature of query cache&lt;/td&gt;
+    &lt;/tr&gt;
+  &lt;/tbody&gt;
+&lt;/table&gt;
+
+&lt;h4 id=&quot;other-optimize-part&quot;&gt;Other optimize part&lt;/h4&gt;
+
+&lt;table&gt;
+  &lt;thead&gt;
+    &lt;tr&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf Key&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf value&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Explanation&lt;/th&gt;
+    &lt;/tr&gt;
+  &lt;/thead&gt;
+  &lt;tbody&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;kylin.query.lazy-query-enabled&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;default false&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;whether to block duplicated sql query&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;kylin.query.lazy-query-waiting-timeout-milliseconds&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;long , in milliseconds, default is 60000&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;max druation for blocking duplicated sql query&lt;/td&gt;
+    &lt;/tr&gt;
+  &lt;/tbody&gt;
+&lt;/table&gt;
+
+&lt;h4 id=&quot;metrics-part&quot;&gt;Metrics part&lt;/h4&gt;
+
+&lt;table&gt;
+  &lt;thead&gt;
+    &lt;tr&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf Key&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf value&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Explanation&lt;/th&gt;
+    &lt;/tr&gt;
+  &lt;/thead&gt;
+  &lt;tbody&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;kylin.metrics.memcached.enabled&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;true&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;Enable memcached metrics in memcached.&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;kylin.metrics.memcached.metricstype&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;off/performance/debug&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;refer to net.spy.memcached.metrics.MetricType&lt;/td&gt;
+    &lt;/tr&gt;
+  &lt;/tbody&gt;
+&lt;/table&gt;
+</description>
+        <pubDate>Tue, 30 Jul 2019 03:30:00 -0700</pubDate>
+        <link>http://kylin.apache.org/blog/2019/07/30/detailed-analysis-of-refine-query-cache/</link>
+        <guid isPermaLink="true">http://kylin.apache.org/blog/2019/07/30/detailed-analysis-of-refine-query-cache/</guid>
+        
+        
+        <category>blog</category>
+        
+      </item>
+    
+      <item>
         <title>Deep dive into Kylin&#39;s Real-time OLAP</title>
         <description>&lt;h2 id=&quot;preface&quot;&gt;Preface&lt;/h2&gt;
 
@@ -921,114 +1191,114 @@ Security: (depend on your security setti
       </item>
     
       <item>
-        <title>Apache Kylin v3.0.0-alpha Release Announcement</title>
-        <description>&lt;p&gt;The Apache Kylin community is pleased to announce the release of Apache Kylin v3.0.0-alpha.&lt;/p&gt;
+        <title>Apache Kylin v3.0.0-alpha 发布</title>
+        <description>&lt;p&gt;近日 Apache Kylin 社区很高兴地宣布,Apache Kylin v3.0.0-alpha 正式发布。&lt;/p&gt;
 
-&lt;p&gt;Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Big Data supporting extremely large datasets.&lt;/p&gt;
+&lt;p&gt;Apache Kylin 是一个开源的分布式分析引擎,旨在为极大数据集提供 SQL 接口和多维分析(OLAP)的能力。&lt;/p&gt;
 
-&lt;p&gt;This is the first release of the new generation v3.x, the main feature introduced is the Real-time OLAP. All of the changes can be found in the &lt;a href=&quot;/docs/release_notes.html&quot;&gt;release notes&lt;/a&gt;. Here we just highlight the main features.&lt;/p&gt;
+&lt;p&gt;这是 Kylin 下一代 v3.x 的第一个发布版本,用于早期预览,主要的功能是实时 (Real-time) OLAP。完整的改动列表请参见&lt;a href=&quot;/docs/release_notes.html&quot;&gt;release notes&lt;/a&gt;;这里挑一些主要改进做说明。&lt;/p&gt;
 
-&lt;h1 id=&quot;important-features&quot;&gt;Important features&lt;/h1&gt;
+&lt;h1 id=&quot;section&quot;&gt;重要新功能&lt;/h1&gt;
 
-&lt;h3 id=&quot;kylin-3654---real-time-olap&quot;&gt;KYLIN-3654 - Real-time OLAP&lt;/h3&gt;
-&lt;p&gt;With the newly introduced Kylin real-time receiver and coordinator components, Kylin can implement a millisecond-level data preparation delay for streaming data from sources like Apache Kafka. This means since v3.0 on,  Kylin can support sub-second level OLAP over historical batch data, near real-time streaming as well as real-time streaming. The user can use one OLAP platform to serve different scenarios. This solution has been deployed and verified in early adopters like eBay since 2018. For how to enable it, please refer to &lt;a href=&quot;/docs30/tutorial/realtime_olap.html&quot;&gt;this tutorial&lt;/a&gt;.&lt;/p&gt;
+&lt;h3 id=&quot;kylin-3654----olap&quot;&gt;KYLIN-3654 - 实时 OLAP&lt;/h3&gt;
+&lt;p&gt;随着引入新的 real-time receiver 和 coordinator 组件,Kylin 能够实现毫秒级别的数据准备延迟,数据源来自流式数据如 Apache Kafka。这意味着,从 v3.0 开始,Kylin 既能够支持历史批量数据的 OLAP,也支持对流式数据的准实时(Near real-time)以及完全实时(real-time)分析。用户可以使用一个 OLAP 平台来服务不同的使用场景。此方案已经在早期用户如 eBay 得到部署和验证。关于如何使用此功能,请参考&lt;a href=&quot;/docs30/tutorial/realtime_olap.html&quot;&gt;æ­¤æ•
 ™ç¨‹&lt;/a&gt;。&lt;/p&gt;
 
-&lt;h3 id=&quot;kylin-3795---submit-spark-jobs-via-apache-livy&quot;&gt;KYLIN-3795 - Submit Spark jobs via Apache Livy&lt;/h3&gt;
-&lt;p&gt;This feature allows the administrator to configure Kylin to integrate with Apache Livy (incubating) for Spark job submissions. The Spark job is submitted to the Livy Server through Livy’s REST API, instead of starting the Spark Driver process in local, which facilitates the management and monitoring of the Spark resources, and also releases the pressure of the nodes where the Kylin job server is running.&lt;/p&gt;
+&lt;h3 id=&quot;kylin-3795----apache-livy--spark-&quot;&gt;KYLIN-3795 - 通过 Apache Livy 递交 Spark 任务&lt;/h3&gt;
+&lt;p&gt;这个功能允许管理员为 Kylin 配置使用 Apache Livy (incubating) 来完成任务的递交。Spark 作业的提交通过 Livy 的 REST API 来提交,而无需在本地启动 Spark Driver 进程,从而方便对 Spark 资源的管理监控,同时也降低对 Kylin 任务进程所在节点的压力。&lt;/p&gt;
 
-&lt;h3 id=&quot;kylin-3820---a-curator-based-job-scheduler&quot;&gt;KYLIN-3820 - A curator-based job scheduler&lt;/h3&gt;
-&lt;p&gt;A new job scheduler is added to automatically discover the Kylin nodes and do an automatic leader selection among them (only the leader will submit jobs). With this feature, you can easily deploy and scale out Kylin nodes without manually update the node address in &lt;code class=&quot;highlighter-rouge&quot;&gt;kylin.properties&lt;/code&gt; and restart Kylin to take effective.&lt;/p&gt;
+&lt;h3 id=&quot;kylin-3820----curator-&quot;&gt;KYLIN-3820 - 基于 Curator 的任务节点分配和服务发现&lt;/h3&gt;
+&lt;p&gt;新增一种基于Apache Zookeeper 和 Curator作业调度器,可以自动发现 Kylin 节点,并自动分配一个节点来进行任务的管理以及故障恢复。有了这个功能后,管理员可以更加容易地部署和扩展 Kylin 节点,而不再需要在 &lt;code class=&quot;highlighter-rouge&quot;&gt;kylin.properties&lt;/code&gt; 中配置每个 Kylin 节点的地址并重启 Kylin 以使之生效。&lt;/p&gt;
 
-&lt;h1 id=&quot;other-enhancements&quot;&gt;Other enhancements&lt;/h1&gt;
+&lt;h1 id=&quot;section-1&quot;&gt;其它改进&lt;/h1&gt;
 
-&lt;h3 id=&quot;kylin-3716---fastthreadlocal-replaces-threadlocal&quot;&gt;KYLIN-3716 - FastThreadLocal replaces ThreadLocal&lt;/h3&gt;
-&lt;p&gt;Using FastThreadLocal instead of ThreadLocal can improve Kylin’s overall performance to some extent.&lt;/p&gt;
+&lt;h3 id=&quot;kylin-3716---fastthreadlocal--threadlocal&quot;&gt;KYLIN-3716 - FastThreadLocal 替换 ThreadLocal&lt;/h3&gt;
+&lt;p&gt;使用 Netty 中的 FastThreadLocal 替代 JDK 原生的 ThreadLocal,可以一定程度上提升 Kylin 在高并发下的性能。&lt;/p&gt;
 
 &lt;h3 id=&quot;kylin-3867---enable-jdbc-to-use-key-store--trust-store-for-https-connection&quot;&gt;KYLIN-3867 - Enable JDBC to use key store &amp;amp; trust store for https connection&lt;/h3&gt;
-&lt;p&gt;By using HTTPS, the authentication information used by JDBC is protected, making Kylin more secure.&lt;/p&gt;
+&lt;p&gt;通过使用HTTPS,保护了JDBC使用的身份验证信息,使得Kylin更加安全&lt;/p&gt;
 
 &lt;h3 id=&quot;kylin-3905---enable-shrunken-dictionary-default&quot;&gt;KYLIN-3905 - Enable shrunken dictionary default&lt;/h3&gt;
-&lt;p&gt;By default, the shrunken dictionary is enabled, and the precise counting scene for high cardinal dimensions can significantly reduce the build time.&lt;/p&gt;
+&lt;p&gt;默认开启 shrunken dictionary,针对高基维进行精确去重的场景,可以显著减少构建用时。&lt;/p&gt;
 
 &lt;h3 id=&quot;kylin-3839---storage-clean-up-after-the-refreshing-and-deleting-a-segment&quot;&gt;KYLIN-3839 - Storage clean up after the refreshing and deleting a segment&lt;/h3&gt;
-&lt;p&gt;Clear unnecessary data files in a timely manner&lt;/p&gt;
+&lt;p&gt;更加及时地清除不必要的数据文件&lt;/p&gt;
 
-&lt;p&gt;&lt;strong&gt;Download&lt;/strong&gt;&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;下载&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;To download Apache Kylin v3.0.0-alpha source code or binary package, visit the &lt;a href=&quot;http://kylin.apache.org/download&quot;&gt;download&lt;/a&gt; page.&lt;/p&gt;
+&lt;p&gt;要下载Apache Kylin 源代码或二进制包,请访问&lt;a href=&quot;/download&quot;&gt;下载页面&lt;/a&gt; page.&lt;/p&gt;
 
-&lt;p&gt;&lt;strong&gt;Upgrade&lt;/strong&gt;&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;升级&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;Follow the &lt;a href=&quot;/docs/howto/howto_upgrade.html&quot;&gt;upgrade guide&lt;/a&gt;.&lt;/p&gt;
+&lt;p&gt;参考&lt;a href=&quot;/docs/howto/howto_upgrade.html&quot;&gt;升级指南&lt;/a&gt;.&lt;/p&gt;
 
-&lt;p&gt;&lt;strong&gt;Feedback&lt;/strong&gt;&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;反馈&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;If you face issue or question, please send mail to Apache Kylin dev or user mailing list: dev@kylin.apache.org , user@kylin.apache.org; Before sending, please make sure you have subscribed the mailing list by dropping an email to dev-subscribe@kylin.apache.org or user-subscribe@kylin.apache.org.&lt;/p&gt;
+&lt;p&gt;如果您遇到问题或疑问,请发送邮件至 Apache Kylin dev 或 user 邮件列表:dev@kylin.apache.org,user@kylin.apache.org; 在发送之前,请确保您已通过发送电子邮件至 dev-subscribe@kylin.apache.org 或 user-subscribe@kylin.apache.org 订阅了邮件列表。&lt;/p&gt;
 
-&lt;p&gt;&lt;em&gt;Great thanks to everyone who contributed!&lt;/em&gt;&lt;/p&gt;
+&lt;p&gt;&lt;em&gt;非常感谢所有贡献Apache Kylin的朋友!&lt;/em&gt;&lt;/p&gt;
 </description>
         <pubDate>Fri, 19 Apr 2019 13:00:00 -0700</pubDate>
-        <link>http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</link>
-        <guid isPermaLink="true">http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</guid>
+        <link>http://kylin.apache.org/cn_blog/2019/04/19/release-v3.0.0-alpha/</link>
+        <guid isPermaLink="true">http://kylin.apache.org/cn_blog/2019/04/19/release-v3.0.0-alpha/</guid>
         
         
-        <category>blog</category>
+        <category>cn_blog</category>
         
       </item>
     
       <item>
-        <title>Apache Kylin v3.0.0-alpha 发布</title>
-        <description>&lt;p&gt;近日 Apache Kylin 社区很高兴地宣布,Apache Kylin v3.0.0-alpha 正式发布。&lt;/p&gt;
+        <title>Apache Kylin v3.0.0-alpha Release Announcement</title>
+        <description>&lt;p&gt;The Apache Kylin community is pleased to announce the release of Apache Kylin v3.0.0-alpha.&lt;/p&gt;
 
-&lt;p&gt;Apache Kylin 是一个开源的分布式分析引擎,旨在为极大数据集提供 SQL 接口和多维分析(OLAP)的能力。&lt;/p&gt;
+&lt;p&gt;Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Big Data supporting extremely large datasets.&lt;/p&gt;
 
-&lt;p&gt;这是 Kylin 下一代 v3.x 的第一个发布版本,用于早期预览,主要的功能是实时 (Real-time) OLAP。完整的改动列表请参见&lt;a href=&quot;/docs/release_notes.html&quot;&gt;release notes&lt;/a&gt;;这里挑一些主要改进做说明。&lt;/p&gt;
+&lt;p&gt;This is the first release of the new generation v3.x, the main feature introduced is the Real-time OLAP. All of the changes can be found in the &lt;a href=&quot;/docs/release_notes.html&quot;&gt;release notes&lt;/a&gt;. Here we just highlight the main features.&lt;/p&gt;
 
-&lt;h1 id=&quot;section&quot;&gt;重要新功能&lt;/h1&gt;
+&lt;h1 id=&quot;important-features&quot;&gt;Important features&lt;/h1&gt;
 
-&lt;h3 id=&quot;kylin-3654----olap&quot;&gt;KYLIN-3654 - 实时 OLAP&lt;/h3&gt;
-&lt;p&gt;随着引入新的 real-time receiver 和 coordinator 组件,Kylin 能够实现毫秒级别的数据准备延迟,数据源来自流式数据如 Apache Kafka。这意味着,从 v3.0 开始,Kylin 既能够支持历史批量数据的 OLAP,也支持对流式数据的准实时(Near real-time)以及完全实时(real-time)分析。用户可以使用一个 OLAP 平台来服务不同的使用场景。此方案已经在早期用户如 eBay 得到部署和验证。关于如何使用此功能,请参考&lt;a href=&quot;/docs30/tutorial/realtime_olap.html&quot;&gt;æ­¤æ•
 ™ç¨‹&lt;/a&gt;。&lt;/p&gt;
+&lt;h3 id=&quot;kylin-3654---real-time-olap&quot;&gt;KYLIN-3654 - Real-time OLAP&lt;/h3&gt;
+&lt;p&gt;With the newly introduced Kylin real-time receiver and coordinator components, Kylin can implement a millisecond-level data preparation delay for streaming data from sources like Apache Kafka. This means since v3.0 on,  Kylin can support sub-second level OLAP over historical batch data, near real-time streaming as well as real-time streaming. The user can use one OLAP platform to serve different scenarios. This solution has been deployed and verified in early adopters like eBay since 2018. For how to enable it, please refer to &lt;a href=&quot;/docs30/tutorial/realtime_olap.html&quot;&gt;this tutorial&lt;/a&gt;.&lt;/p&gt;
 
-&lt;h3 id=&quot;kylin-3795----apache-livy--spark-&quot;&gt;KYLIN-3795 - 通过 Apache Livy 递交 Spark 任务&lt;/h3&gt;
-&lt;p&gt;这个功能允许管理员为 Kylin 配置使用 Apache Livy (incubating) 来完成任务的递交。Spark 作业的提交通过 Livy 的 REST API 来提交,而无需在本地启动 Spark Driver 进程,从而方便对 Spark 资源的管理监控,同时也降低对 Kylin 任务进程所在节点的压力。&lt;/p&gt;
+&lt;h3 id=&quot;kylin-3795---submit-spark-jobs-via-apache-livy&quot;&gt;KYLIN-3795 - Submit Spark jobs via Apache Livy&lt;/h3&gt;
+&lt;p&gt;This feature allows the administrator to configure Kylin to integrate with Apache Livy (incubating) for Spark job submissions. The Spark job is submitted to the Livy Server through Livy’s REST API, instead of starting the Spark Driver process in local, which facilitates the management and monitoring of the Spark resources, and also releases the pressure of the nodes where the Kylin job server is running.&lt;/p&gt;
 
-&lt;h3 id=&quot;kylin-3820----curator-&quot;&gt;KYLIN-3820 - 基于 Curator 的任务节点分配和服务发现&lt;/h3&gt;
-&lt;p&gt;新增一种基于Apache Zookeeper 和 Curator作业调度器,可以自动发现 Kylin 节点,并自动分配一个节点来进行任务的管理以及故障恢复。有了这个功能后,管理员可以更加容易地部署和扩展 Kylin 节点,而不再需要在 &lt;code class=&quot;highlighter-rouge&quot;&gt;kylin.properties&lt;/code&gt; 中配置每个 Kylin 节点的地址并重启 Kylin 以使之生效。&lt;/p&gt;
+&lt;h3 id=&quot;kylin-3820---a-curator-based-job-scheduler&quot;&gt;KYLIN-3820 - A curator-based job scheduler&lt;/h3&gt;
+&lt;p&gt;A new job scheduler is added to automatically discover the Kylin nodes and do an automatic leader selection among them (only the leader will submit jobs). With this feature, you can easily deploy and scale out Kylin nodes without manually update the node address in &lt;code class=&quot;highlighter-rouge&quot;&gt;kylin.properties&lt;/code&gt; and restart Kylin to take effective.&lt;/p&gt;
 
-&lt;h1 id=&quot;section-1&quot;&gt;其它改进&lt;/h1&gt;
+&lt;h1 id=&quot;other-enhancements&quot;&gt;Other enhancements&lt;/h1&gt;
 
-&lt;h3 id=&quot;kylin-3716---fastthreadlocal--threadlocal&quot;&gt;KYLIN-3716 - FastThreadLocal 替换 ThreadLocal&lt;/h3&gt;
-&lt;p&gt;使用 Netty 中的 FastThreadLocal 替代 JDK 原生的 ThreadLocal,可以一定程度上提升 Kylin 在高并发下的性能。&lt;/p&gt;
+&lt;h3 id=&quot;kylin-3716---fastthreadlocal-replaces-threadlocal&quot;&gt;KYLIN-3716 - FastThreadLocal replaces ThreadLocal&lt;/h3&gt;
+&lt;p&gt;Using FastThreadLocal instead of ThreadLocal can improve Kylin’s overall performance to some extent.&lt;/p&gt;
 
 &lt;h3 id=&quot;kylin-3867---enable-jdbc-to-use-key-store--trust-store-for-https-connection&quot;&gt;KYLIN-3867 - Enable JDBC to use key store &amp;amp; trust store for https connection&lt;/h3&gt;
-&lt;p&gt;通过使用HTTPS,保护了JDBC使用的身份验证信息,使得Kylin更加安全&lt;/p&gt;
+&lt;p&gt;By using HTTPS, the authentication information used by JDBC is protected, making Kylin more secure.&lt;/p&gt;
 
 &lt;h3 id=&quot;kylin-3905---enable-shrunken-dictionary-default&quot;&gt;KYLIN-3905 - Enable shrunken dictionary default&lt;/h3&gt;
-&lt;p&gt;默认开启 shrunken dictionary,针对高基维进行精确去重的场景,可以显著减少构建用时。&lt;/p&gt;
+&lt;p&gt;By default, the shrunken dictionary is enabled, and the precise counting scene for high cardinal dimensions can significantly reduce the build time.&lt;/p&gt;
 
 &lt;h3 id=&quot;kylin-3839---storage-clean-up-after-the-refreshing-and-deleting-a-segment&quot;&gt;KYLIN-3839 - Storage clean up after the refreshing and deleting a segment&lt;/h3&gt;
-&lt;p&gt;更加及时地清除不必要的数据文件&lt;/p&gt;
+&lt;p&gt;Clear unnecessary data files in a timely manner&lt;/p&gt;
 
-&lt;p&gt;&lt;strong&gt;下载&lt;/strong&gt;&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;Download&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;要下载Apache Kylin 源代码或二进制包,请访问&lt;a href=&quot;/download&quot;&gt;下载页面&lt;/a&gt; page.&lt;/p&gt;
+&lt;p&gt;To download Apache Kylin v3.0.0-alpha source code or binary package, visit the &lt;a href=&quot;http://kylin.apache.org/download&quot;&gt;download&lt;/a&gt; page.&lt;/p&gt;
 
-&lt;p&gt;&lt;strong&gt;升级&lt;/strong&gt;&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;Upgrade&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;参考&lt;a href=&quot;/docs/howto/howto_upgrade.html&quot;&gt;升级指南&lt;/a&gt;.&lt;/p&gt;
+&lt;p&gt;Follow the &lt;a href=&quot;/docs/howto/howto_upgrade.html&quot;&gt;upgrade guide&lt;/a&gt;.&lt;/p&gt;
 
-&lt;p&gt;&lt;strong&gt;反馈&lt;/strong&gt;&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;Feedback&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;如果您遇到问题或疑问,请发送邮件至 Apache Kylin dev 或 user 邮件列表:dev@kylin.apache.org,user@kylin.apache.org; 在发送之前,请确保您已通过发送电子邮件至 dev-subscribe@kylin.apache.org 或 user-subscribe@kylin.apache.org 订阅了邮件列表。&lt;/p&gt;
+&lt;p&gt;If you face issue or question, please send mail to Apache Kylin dev or user mailing list: dev@kylin.apache.org , user@kylin.apache.org; Before sending, please make sure you have subscribed the mailing list by dropping an email to dev-subscribe@kylin.apache.org or user-subscribe@kylin.apache.org.&lt;/p&gt;
 
-&lt;p&gt;&lt;em&gt;非常感谢所有贡献Apache Kylin的朋友!&lt;/em&gt;&lt;/p&gt;
+&lt;p&gt;&lt;em&gt;Great thanks to everyone who contributed!&lt;/em&gt;&lt;/p&gt;
 </description>
         <pubDate>Fri, 19 Apr 2019 13:00:00 -0700</pubDate>
-        <link>http://kylin.apache.org/cn_blog/2019/04/19/release-v3.0.0-alpha/</link>
-        <guid isPermaLink="true">http://kylin.apache.org/cn_blog/2019/04/19/release-v3.0.0-alpha/</guid>
+        <link>http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</link>
+        <guid isPermaLink="true">http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</guid>
         
         
-        <category>cn_blog</category>
+        <category>blog</category>
         
       </item>
     
@@ -1185,85 +1455,6 @@ The checkpoint info is the smallest part
       </item>
     
       <item>
-        <title>Apache Kylin v2.6.0 正式发布</title>
-        <description>&lt;p&gt;近日Apache Kylin 社区很高兴地宣布,Apache Kylin 2.6.0 正式发布。&lt;/p&gt;
-
-&lt;p&gt;Apache Kylin 是一个开源的分布式分析引擎,旨在为极大数据集提供 SQL 接口和多维分析(OLAP)的能力。&lt;/p&gt;
-
-&lt;p&gt;这是继2.5.0 后的一个新功能版本。该版本引入了很多有价值的改进,完整的改动列表请参见&lt;a href=&quot;https://kylin.apache.org/docs/release_notes.html&quot;&gt;release notes&lt;/a&gt;;这里挑一些主要改进做说明:&lt;/p&gt;
-
-&lt;h3 id=&quot;jdbcsdk&quot;&gt;针对以JDBC为数据源的SDK&lt;/h3&gt;
-&lt;p&gt;Kylin目前已经支持通过JDBC连接包括Amazon Redshift, SQL Server在内的多种数据源。&lt;br /&gt;
-为了便于开发者更便利地处理各种 SQL 方言(dialect) 的不同以更加简单地开发新的 JDBC 数据源,Kylin 提供了相应的 SDK 和统一的 API 入口:&lt;br /&gt;
-* 同步元数据和数据&lt;br /&gt;
-* 构建 cube&lt;br /&gt;
-* 当找不到相应的cube来解答查询时,下推查询到数据源&lt;/p&gt;
-
-&lt;p&gt;更多内容参见 KYLIN-3552。&lt;/p&gt;
-
-&lt;h3 id=&quot;memcached--kylin-&quot;&gt;使用 Memcached 作 Kylin 的分布式缓存&lt;/h3&gt;
-&lt;p&gt;在过去,Kylin 对查询结果的缓存不是十分高效,主要有以下两个方面的原因:&lt;br /&gt;
-一个是当 Kylin 的 metadata 发生变化时,会主动盲目地去清除大量缓存,使得缓存会被频繁刷新而导致利用率降低。&lt;br /&gt;
-另一点是由于只使用本地缓存而导致 Kylin server 之间不能共享彼此的缓存,这样查询的缓存命中率就会降低。&lt;br /&gt;
-本地缓存的一个缺点是大小受到限制,不能像分布式缓存那样水平扩展。这样导致能缓存的查询结果量受到了限制。&lt;/p&gt;
-
-&lt;p&gt;针对这些缺陷,我们改变了缓存失效的机制,不再主动去清理缓存,而是采取如下的方案:&lt;br /&gt;
-1. 在将查询结果放入缓存之前,根据当前的元数据信息计算一个数字签名,并与查询结果一同放入缓存中&lt;br /&gt;
-2. 从缓存中获取查询结果之后,根据当前的元数据信息计算一个数字签名,对比两者的数字签名是否一致。如果一致,那么缓存有效;反之,该缓存失效并删除&lt;/p&gt;
-
-&lt;p&gt;我们还引入了 Memcached 作为 Kylin 的分布式缓存。这样 Kylin server 之间可以共享查询结果的缓存,而且由于 Memcached server 之间的独立性,非常易于水平拓展,更加有利于缓存更多的数据。&lt;br /&gt;
-相关开发任务是 KYLIN-2895, KYLIN-2894, KYLIN-2896, KYLIN-2897, KYLIN-2898, KYLIN-2899。&lt;/p&gt;
-
-&lt;h3 id=&quot;forkjoinpool--fast-cubing-&quot;&gt;ForkJoinPool 简化 fast cubing 的线程模型&lt;/h3&gt;
-&lt;p&gt;在过去进行 fast cubing 时,Kylin 使用自己定义的一系列线程,如 split 线程,task 线程,main 线程等等进行并发的 cube 构建。&lt;br /&gt;
-在这个线程模型中,线程之间的关系十分的复杂,而且对异常处理也十分容易出错。&lt;/p&gt;
-
-&lt;p&gt;现在我们引入了 ForkJoinPool,在主线程中处理 split 逻辑,构建 cuboid 的任务以及子任务都在 fork join pool中执行,cuboid 构建的结果可以被异步的收集并且可以更早地输出给下游的 merge 操作。更多内容参见 KYLIN-2932。&lt;/p&gt;
-
-&lt;h3 id=&quot;hllcounter-&quot;&gt;改进 HLLCounter 的性能&lt;/h3&gt;
-&lt;p&gt;对于 HLLCounter, 我们从两方面进行了改进:构建 HLLCounter 和计算调和平均的方式。&lt;br /&gt;
-1. 关于 HLLCounter 的构建,我们不再使用merge的方式,而是直接copy别的HLLCounter里面的registers&lt;br /&gt;
-2. 关于计算 HLLCSnapshot 里面的调和平均,做了以下三个方面的改进:&lt;br /&gt;
-* 缓存所有的1/2^r&lt;br /&gt;
-* 使用整型相加代替浮点型相加&lt;br /&gt;
-* 删除条件分支,例如无需检查 registers[i] 是不是为0&lt;/p&gt;
-
-&lt;p&gt;更多内容参见 KYLIN-3656。&lt;/p&gt;
-
-&lt;h3 id=&quot;cube-planner-&quot;&gt;改进 Cube Planner 算法&lt;/h3&gt;
-&lt;p&gt;在过去,cube planner 的 phase two 增加未被预计算的 cuboid 的方式只能通过 mandatory cuboid 的方式。而一个 cuboid 是否为 mandatory,又有两种方式:&lt;br /&gt;
-手动设置,查询时 rollup 的行数足够大。这里通过判断查询时 rollup 的行数是否足够大来判断是否为 mandatory cuboid 的方式有两大缺陷:&lt;br /&gt;
-* 一是估算 rollup 的行数的算法不是很好&lt;br /&gt;
-* 二是很难设立一个静态的阈值来做判定&lt;/p&gt;
-
-&lt;p&gt;现在我们不再从 rollup 行数的角度看问题了。一切都是从 cuboid 行数的角度看问题,这样就和 cost based 的 cube planner 算法做了统一。&lt;br /&gt;
-为此我们通过使用 rollup 比率来改进了未被预先构建的 cuboid 的行数的估算,然后让 cost based 的 cube planner 算法来判定哪些未被构建的 cuboid 该被构建,哪些该被遗弃。&lt;br /&gt;
-通过这样的改进,无需通过设定静态的阈值来推荐 mandatory cuboid 了,而 mandatory cuboid 只能被手动设置,不能被推荐了。更多内容参见 KYLIN-3540。&lt;/p&gt;
-
-&lt;p&gt;&lt;strong&gt;下载&lt;/strong&gt;&lt;/p&gt;
-
-&lt;p&gt;要下载Apache Kylin v2.6.0源代码或二进制包,请访问&lt;a href=&quot;http://kylin.apache.org/download&quot;&gt;下载页面&lt;/a&gt; .&lt;/p&gt;
-
-&lt;p&gt;&lt;strong&gt;升级&lt;/strong&gt;&lt;/p&gt;
-
-&lt;p&gt;参考&lt;a href=&quot;/docs/howto/howto_upgrade.html&quot;&gt;升级指南&lt;/a&gt;.&lt;/p&gt;
-
-&lt;p&gt;&lt;strong&gt;反馈&lt;/strong&gt;&lt;/p&gt;
-
-&lt;p&gt;如果您遇到问题或疑问,请发送邮件至 Apache Kylin dev 或 user 邮件列表:dev@kylin.apache.org,user@kylin.apache.org; 在发送之前,请确保您已通过发送电子邮件至 dev-subscribe@kylin.apache.org 或 user-subscribe@kylin.apache.org订阅了邮件列表。&lt;/p&gt;
-
-&lt;p&gt;&lt;em&gt;非常感谢所有贡献Apache Kylin的朋友!&lt;/em&gt;&lt;/p&gt;
-</description>
-        <pubDate>Fri, 18 Jan 2019 12:00:00 -0800</pubDate>
-        <link>http://kylin.apache.org/cn_blog/2019/01/18/release-v2.6.0/</link>
-        <guid isPermaLink="true">http://kylin.apache.org/cn_blog/2019/01/18/release-v2.6.0/</guid>
-        
-        
-        <category>cn_blog</category>
-        
-      </item>
-    
-      <item>
         <title>Apache Kylin v2.6.0 Release Announcement</title>
         <description>&lt;p&gt;The Apache Kylin community is pleased to announce the release of Apache Kylin v2.6.0.&lt;/p&gt;
 
@@ -1342,274 +1533,81 @@ By this improvement, we don’t need
       </item>
     
       <item>
-        <title>How Cisco&#39;s Big Data Team Improved the High Concurrent Throughput of Apache Kylin by 5x</title>
-        <description>&lt;h2 id=&quot;background&quot;&gt;Background&lt;/h2&gt;
-
-&lt;p&gt;As part of the development group of Cisco’s Big Data team, one of our responsibilities is to provide BI reports to our stakeholders. Stakeholders rely on the reporting system to check the usage of Cisco’s business offerings. These reports are also used as a reference for billing, so they are critical to our stakeholders and the business overall.&lt;/p&gt;
-
-&lt;p&gt;The raw data for these reports is sourced across multiple tables in our Oracle database. The monthly data volume for one table is in the billions, and if a customer wants to run a report for one year, at least one billion to two billion rows of data need to be aggregated or processed through other operations. Additionally, all results need to be provided in a short amount of time. In the course of our research, we discovered Apache Kylin, a distributed preprocessing engine for massive datasets based on pre-calculation, which enables you to query those massive datasets at sub-second latency.&lt;/p&gt;
-
-&lt;p&gt;With the simulation test using our production data, we found that Kylin was ideal for our needs and was indeed capable of providing aggregated results on one billion rows of data in one second. However, we still needed to undergo additional tests for another use case. For one stakeholder, we provide 15 charts displayed on a single page. The BI system will send REST API requests to Kylin to query the data for each chart asynchronously. Based on the production data volume, if there are 20 stakeholders viewing the report on one node, they will trigger 15*20 = 300 requests. The high concurrent query performance of Kylin is what we needed to test here.&lt;/p&gt;
-
-&lt;h2 id=&quot;the-testing-stage&quot;&gt;The Testing Stage&lt;/h2&gt;
-
-&lt;p&gt;&lt;strong&gt;Precondition&lt;/strong&gt;: To reduce the impact from network cost, we deployed the testing tools in the same network environment with Kylin. Meanwhile, we turned off the query cache for Kylin to make sure that each request was executed on the bottom layer.&lt;/p&gt;
-
-&lt;p&gt;&lt;strong&gt;Testing Tools&lt;/strong&gt;: Aside from our traditional testing tool, Apache Jmeter, we also used another open source tool: Gatlin (&lt;a href=&quot;https://gatling.io/&quot;&gt;https://gatling.io/&lt;/a&gt;) to test the same case. We excluded the impact from the tools.&lt;/p&gt;
-
-&lt;p&gt;&lt;strong&gt;Testing Strategy&lt;/strong&gt;: We simulated user requests of different sizes by increasing the number of concurrent threads, tracking the average response in 60 seconds, finding the bottleneck for Kylin query responses, and observing the maximum response time and success rate.&lt;/p&gt;
-
-&lt;p&gt;&lt;strong&gt;Testing Results&lt;/strong&gt;:&lt;/p&gt;
-
-&lt;table&gt;
-  &lt;thead&gt;
-    &lt;tr&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Thread&lt;/th&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Handled Queries (in 60 seconds)&lt;/th&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Handled Queries (per second)&lt;/th&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Mean Response Time (ms)&lt;/th&gt;
-    &lt;/tr&gt;
-  &lt;/thead&gt;
-  &lt;tbody&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;1&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;773&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;13&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;77&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;15&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;3245&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;54&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;279&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;25&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;3844&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;64&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;390&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;50&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;4912&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;82&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;612&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;75&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;5405&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;90&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;841&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;100&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;5436&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;91&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;1108&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;150&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;5434&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;91&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;1688&lt;/td&gt;
-    &lt;/tr&gt;
-  &lt;/tbody&gt;
-&lt;/table&gt;
-
-&lt;p&gt;Resulting in the line chart as follows:&lt;/p&gt;
-
-&lt;p&gt;&lt;img src=&quot;/images/blog/cisco_throughput_5x/handled_queries_1.png&quot; alt=&quot;&quot; width=&quot;500px&quot; height=&quot;300px&quot; /&gt;&lt;/p&gt;
-
-&lt;p&gt;​&lt;/p&gt;
-
-&lt;p&gt;&lt;strong&gt;Finding&lt;/strong&gt;: When the number of concurrent threads reach 75, executed queries per second reach a peak of 90. The number does not become better even as we continue to increase the threads. 90 concurrent query responses in one second only allows 90/15 = 6 users to view a report at the same time. Even when we extend the Kylin query nodes to 3, query capability with 18 users per second is far behind our business demands.&lt;/p&gt;
-
-&lt;h2 id=&quot;root-cause-analysis&quot;&gt;Root Cause Analysis&lt;/h2&gt;
-
-&lt;p&gt;After reading and analyzing the query engine code of Kylin, we learned that Kylin’s query performs parallel filtering and calculation in HBase’s region server by launching HBase Coprocessor. Based on this information, we checked the resource usage of the HBase cluster. The count of RPC Tasks processed on the region server did not increase linearly with the number of Kylin query requests when high concurrent queries occured. We concluded that there was a thread block on the Kylin side.&lt;/p&gt;
-
-&lt;p&gt;We used Flame graph and JProfile to collect and analyze data from the Kylin query node and could not find the root cause. Then we tried to catch a thread snapshot of Kylin with Jstack. Analyzing the Jstack log, we discovered the root cause of the bottleneck causing this concurrent query issue. The example is as follows (Kylin version 2.5.0):&lt;/p&gt;
-
-&lt;p&gt;One thread is locked at sun.misc.URLClassPath.getNextLoader. TID is 0x000000048007a180:&lt;/p&gt;
-
-&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&quot;Query e9c44a2d-6226-ff3b-f984-ce8489107d79-3425&quot; #3425 daemon prio=5 os_prio=0 tid=0x000000000472b000 nid=0x1433 waiting ]
-   java.lang.Thread.State: BLOCKED (on object monitor)
-    at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:469)
-    - locked &amp;lt;0x000000048007a180&amp;gt; (a sun.misc.URLClassPath)
-    at sun.misc.URLClassPath.findResource(URLClassPath.java:214)
-    at java.net.URLClassLoader$2.run(URLClassLoader.java:569)
-    at java.net.URLClassLoader$2.run(URLClassLoader.java:567)
-    at java.security.AccessController.doPrivileged(Native Method)
-    at java.net.URLClassLoader.findResource(URLClassLoader.java:566)
-    at java.lang.ClassLoader.getResource(ClassLoader.java:1096)
-    at java.lang.ClassLoader.getResource(ClassLoader.java:1091)
-    at org.apache.catalina.loader.WebappClassLoaderBase.getResource(WebappClassLoaderBase.java:1666)
-    at org.apache.kylin.common.KylinConfig.buildSiteOrderedProps(KylinConfig.java:338)
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;p&gt;43 threads were waiting to lock &amp;lt;0x000000048007a180&amp;gt;  at the same time:&lt;/p&gt;
-
-&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&quot;Query f1f0bbec-a3f7-04b2-1ac6-fd3e03a0232d-4002&quot; #4002 daemon prio=5 os_prio=0 tid=0x00007f27e71e7800 nid=0x1676 waiting ]
-   java.lang.Thread.State: BLOCKED (on object monitor)
-    at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:469)
-    - waiting to lock &amp;lt;0x000000048007a180&amp;gt; (a sun.misc.URLClassPath)
-    at sun.misc.URLClassPath.findResource(URLClassPath.java:214)
-    at java.net.URLClassLoader$2.run(URLClassLoader.java:569)
-    at java.net.URLClassLoader$2.run(URLClassLoader.java:567)
-    at java.security.AccessController.doPrivileged(Native Method)
-    at java.net.URLClassLoader.findResource(URLClassLoader.java:566)
-    at java.lang.ClassLoader.getResource(ClassLoader.java:1096)
-    at java.lang.ClassLoader.getResource(ClassLoader.java:1091)
-    at org.apache.catalina.loader.WebappClassLoaderBase.getResource(WebappClassLoaderBase.java:1666)
-    at org.apache.kylin.common.KylinConfig.buildSiteOrderedProps(KylinConfig.java:338)
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;p&gt;We found that the closest code logic to Kylin was:&lt;/p&gt;
-
-&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;org.apache.kylin.common.KylinConfig.buildSiteOrderedProps(KylinConfig.java:338)
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;p&gt;Further analyzing the Kylin source code showed we were getting close to the resolution.&lt;/p&gt;
-
-&lt;h2 id=&quot;code-analysis&quot;&gt;Code Analysis&lt;/h2&gt;
-
-&lt;p&gt;When Kylin query engine builds a request to HBase Coprocessor, it will export Kylin properties (various properties used in Kylin) as Strings. This issue is caused by the relative code logic.&lt;/p&gt;
-
-&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;function private static OrderedProperties buildSiteOrderedProps()
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;ul&gt;
-  &lt;li&gt;Each thread will getResouce to load “kylin-defaults.properties” (the default properties file that users cannot modify).&lt;/li&gt;
-&lt;/ul&gt;
-
-&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;// 1. load default configurations from classpath.
-// we have a kylin-defaults.properties in kylin/core-common/src/main/resources
-URL resource = Thread.currentThread().getContextClassLoader().getResource(&quot;kylin-defaults.properties&quot;);
-Preconditions.checkNotNull(resource);
-logger.info(&quot;Loading kylin-defaults.properties from {}&quot;, resource.getPath());
-OrderedProperties orderedProperties = new OrderedProperties();
-loadPropertiesFromInputStream(resource.openStream(), orderedProperties);
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;ul&gt;
-  &lt;li&gt;Loop 10 times to getResouce for  “kylin-defaults” + (i) + “.properties”. Thread LOCKED occurs here.&lt;/li&gt;
-&lt;/ul&gt;
-
-&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;for (int i = 0; i &amp;lt; 10; i++) {
-String fileName = &quot;kylin-defaults&quot; +  + &quot;.properties&quot;;
- URL additionalResource = Thread.currentThread().getContextClassLoader().getResource(fileName);
- if (additionalResource != null) {
-        logger.info(&quot;Loading {} from {} &quot;, fileName, additionalResource.getPath());
- loadPropertiesFromInputStream(additionalResource.openStream(), orderedProperties);
- }
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
+        <title>Apache Kylin v2.6.0 正式发布</title>
+        <description>&lt;p&gt;近日Apache Kylin 社区很高兴地宣布,Apache Kylin 2.6.0 正式发布。&lt;/p&gt;
 
-&lt;p&gt;Those logics were introduced in 2017/6/7, with JIRA ID KYLIN-2659 &lt;em&gt;Refactor KylinConfig so that all the default configurations are hidden in kylin-defaults.properties&lt;/em&gt; reported by Hongbin Ma.&lt;/p&gt;
+&lt;p&gt;Apache Kylin 是一个开源的分布式分析引擎,旨在为极大数据集提供 SQL 接口和多维分析(OLAP)的能力。&lt;/p&gt;
 
-&lt;h2 id=&quot;issue-fixing&quot;&gt;Issue Fixing&lt;/h2&gt;
+&lt;p&gt;这是继2.5.0 后的一个新功能版本。该版本引入了很多有价值的改进,完整的改动列表请参见&lt;a href=&quot;https://kylin.apache.org/docs/release_notes.html&quot;&gt;release notes&lt;/a&gt;;这里挑一些主要改进做说明:&lt;/p&gt;
 
-&lt;p&gt;For the first part of the logic, because kylin-defaults.properties is built in kylin-core-common-xxxx.jar, there’s no need to getResource for it every time. We moved this logic to getInstanceFromEnv(). This logic gets called only once when service starts.&lt;/p&gt;
+&lt;h3 id=&quot;jdbcsdk&quot;&gt;针对以JDBC为数据源的SDK&lt;/h3&gt;
+&lt;p&gt;Kylin目前已经支持通过JDBC连接包括Amazon Redshift, SQL Server在内的多种数据源。&lt;br /&gt;
+为了便于开发者更便利地处理各种 SQL 方言(dialect) 的不同以更加简单地开发新的 JDBC 数据源,Kylin 提供了相应的 SDK 和统一的 API 入口:&lt;br /&gt;
+* 同步元数据和数据&lt;br /&gt;
+* 构建 cube&lt;br /&gt;
+* 当找不到相应的cube来解答查询时,下推查询到数据源&lt;/p&gt;
 
-&lt;p&gt;We found one regression issue when fixing this bug. One class, CubeVisitService, is a Coprocessor. It will use KylinConfig as util class to generate KylinConfig object. It’s dangerous to induce any logic to load properties. Due to this, there is no Kylin.properties file in Coprocessor.&lt;/p&gt;
+&lt;p&gt;更多内容参见 KYLIN-3552。&lt;/p&gt;
 
-&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;buildDefaultOrderedProperties();
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
+&lt;h3 id=&quot;memcached--kylin-&quot;&gt;使用 Memcached 作 Kylin 的分布式缓存&lt;/h3&gt;
+&lt;p&gt;在过去,Kylin 对查询结果的缓存不是十分高效,主要有以下两个方面的原因:&lt;br /&gt;
+一个是当 Kylin 的 metadata 发生变化时,会主动盲目地去清除大量缓存,使得缓存会被频繁刷新而导致利用率降低。&lt;br /&gt;
+另一点是由于只使用本地缓存而导致 Kylin server 之间不能共享彼此的缓存,这样查询的缓存命中率就会降低。&lt;br /&gt;
+本地缓存的一个缺点是大小受到限制,不能像分布式缓存那样水平扩展。这样导致能缓存的查询结果量受到了限制。&lt;/p&gt;
 
-&lt;p&gt;For the second part, this design should be future-proof and allow users to define 10 default properties (and override with each other), but after a year and a half, this logic seemed to never be used. However, to reduce risk, we kept this logic because it only gets called once during service start up which resulted in an insignificant waste of additional time.&lt;/p&gt;
+&lt;p&gt;针对这些缺陷,我们改变了缓存失效的机制,不再主动去清理缓存,而是采取如下的方案:&lt;br /&gt;
+1. 在将查询结果放入缓存之前,根据当前的元数据信息计算一个数字签名,并与查询结果一同放入缓存中&lt;br /&gt;
+2. 从缓存中获取查询结果之后,根据当前的元数据信息计算一个数字签名,对比两者的数字签名是否一致。如果一致,那么缓存有效;反之,该缓存失效并删除&lt;/p&gt;
 
-&lt;h2 id=&quot;performance-testing-after-bug-fixes&quot;&gt;Performance Testing After Bug Fixes&lt;/h2&gt;
+&lt;p&gt;我们还引入了 Memcached 作为 Kylin 的分布式缓存。这样 Kylin server 之间可以共享查询结果的缓存,而且由于 Memcached server 之间的独立性,非常易于水平拓展,更加有利于缓存更多的数据。&lt;br /&gt;
+相关开发任务是 KYLIN-2895, KYLIN-2894, KYLIN-2896, KYLIN-2897, KYLIN-2898, KYLIN-2899。&lt;/p&gt;
 
-&lt;p&gt;Based on the same data volume and testing environment, results were as follows:&lt;/p&gt;
+&lt;h3 id=&quot;forkjoinpool--fast-cubing-&quot;&gt;ForkJoinPool 简化 fast cubing 的线程模型&lt;/h3&gt;
+&lt;p&gt;在过去进行 fast cubing 时,Kylin 使用自己定义的一系列线程,如 split 线程,task 线程,main 线程等等进行并发的 cube 构建。&lt;br /&gt;
+在这个线程模型中,线程之间的关系十分的复杂,而且对异常处理也十分容易出错。&lt;/p&gt;
 
-&lt;table&gt;
-  &lt;thead&gt;
-    &lt;tr&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Thread&lt;/th&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Handled Queries (in 60 seconds)&lt;/th&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Handled Queries (per second)&lt;/th&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Mean Response Time (ms)&lt;/th&gt;
-    &lt;/tr&gt;
-  &lt;/thead&gt;
-  &lt;tbody&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;1&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;2451&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;41&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;12&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;15&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;12422&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;207&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;37&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;25&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;15600&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;260&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;56&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;50&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;18481&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;308&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;129&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;75&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;21055&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;351&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;136&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;100&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;24036&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;400&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;251&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;150&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;28014&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;467&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;277&lt;/td&gt;
-    &lt;/tr&gt;
-  &lt;/tbody&gt;
-&lt;/table&gt;
+&lt;p&gt;现在我们引入了 ForkJoinPool,在主线程中处理 split 逻辑,构建 cuboid 的任务以及子任务都在 fork join pool中执行,cuboid 构建的结果可以被异步的收集并且可以更早地输出给下游的 merge 操作。更多内容参见 KYLIN-2932。&lt;/p&gt;
 
-&lt;p&gt;And the resulting line chart:&lt;/p&gt;
+&lt;h3 id=&quot;hllcounter-&quot;&gt;改进 HLLCounter 的性能&lt;/h3&gt;
+&lt;p&gt;对于 HLLCounter, 我们从两方面进行了改进:构建 HLLCounter 和计算调和平均的方式。&lt;br /&gt;
+1. 关于 HLLCounter 的构建,我们不再使用merge的方式,而是直接copy别的HLLCounter里面的registers&lt;br /&gt;
+2. 关于计算 HLLCSnapshot 里面的调和平均,做了以下三个方面的改进:&lt;br /&gt;
+* 缓存所有的1/2^r&lt;br /&gt;
+* 使用整型相加代替浮点型相加&lt;br /&gt;
+* 删除条件分支,例如无需检查 registers[i] 是不是为0&lt;/p&gt;
 
-&lt;p&gt;&lt;img src=&quot;/images/blog/cisco_throughput_5x/handled_queries_2.png&quot; alt=&quot;&quot; width=&quot;500px&quot; height=&quot;300px&quot; /&gt;&lt;/p&gt;
+&lt;p&gt;更多内容参见 KYLIN-3656。&lt;/p&gt;
 
-&lt;p&gt;When the concurrent threads reached 150, Kylin processed 467 requests per second. The concurrent query capability increased by five times with linear growth . It could be concluded then that the bottleneck was eliminated . We didn’t increase the concurrent threads due to the Kylin query engine’s settings for cluster load balancing which meant that increasing the concurrent connections on a single node increased the workload on the Tomcat server (Max connection is 150 in Kylin by default). The thread blocking issue disappeared after re-collecting and analyzing the Jstack log.&lt;/p&gt;
+&lt;h3 id=&quot;cube-planner-&quot;&gt;改进 Cube Planner 算法&lt;/h3&gt;
+&lt;p&gt;在过去,cube planner 的 phase two 增加未被预计算的 cuboid 的方式只能通过 mandatory cuboid 的方式。而一个 cuboid 是否为 mandatory,又有两种方式:&lt;br /&gt;
+手动设置,查询时 rollup 的行数足够大。这里通过判断查询时 rollup 的行数是否足够大来判断是否为 mandatory cuboid 的方式有两大缺陷:&lt;br /&gt;
+* 一是估算 rollup 的行数的算法不是很好&lt;br /&gt;
+* 二是很难设立一个静态的阈值来做判定&lt;/p&gt;
 
-&lt;p&gt;After the fix, each Kylin node could now handle requests for 467/15 = 31 users, which meets our business requirement. Additionally, Kylin’s concurrent query capability can be further improved by several times once we enable query cache, so it is more than sufficient to fulfill our needs.&lt;/p&gt;
+&lt;p&gt;现在我们不再从 rollup 行数的角度看问题了。一切都是从 cuboid 行数的角度看问题,这样就和 cost based 的 cube planner 算法做了统一。&lt;br /&gt;
+为此我们通过使用 rollup 比率来改进了未被预先构建的 cuboid 的行数的估算,然后让 cost based 的 cube planner 算法来判定哪些未被构建的 cuboid 该被构建,哪些该被遗弃。&lt;br /&gt;
+通过这样的改进,无需通过设定静态的阈值来推荐 mandatory cuboid 了,而 mandatory cuboid 只能被手动设置,不能被推荐了。更多内容参见 KYLIN-3540。&lt;/p&gt;
 
-&lt;h2 id=&quot;summary&quot;&gt;Summary&lt;/h2&gt;
+&lt;p&gt;&lt;strong&gt;下载&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;Apache Kylin lets you query massive datasets at sub-second latency, thanks to the pre-calculation design of cubes, the optimization of Apache Calcite operator in queries, and also the introduction of “Prepared Statement Cache” to reduce the cost of Calcite SQL parses. Query performance optimization is not easy. We need to pay more attention to impacts on the Kylin query engine when new features are introduced or bugs are fixed, since even a minor code change could spell disaster. Issues like these in high concurrency scenarios can often be hard to reproduce and analyze.&lt;/p&gt;
+&lt;p&gt;要下载Apache Kylin v2.6.0源代码或二进制包,请访问&lt;a href=&quot;http://kylin.apache.org/download&quot;&gt;下载页面&lt;/a&gt; .&lt;/p&gt;
 
-&lt;p&gt;Lastly, query performance testing should not be limited to a single or small set of queries. High concurrecy performance testing should take place considering actual business requirements. For enterprise reporting systems, 3 seconds is the user tolerance limit for new page loading, which includes page rendering and network consumption. Ultimately, the backend data service should provide a response within 1 second. This is indeed a big challenge in a business scenario with big data sets. Fortunately, Kylin easily meets this requirement.&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;升级&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;This issue has already been submitted on JIRA as &lt;a href=&quot;https://issues.apache.org/jira/browse/KYLIN-3672&quot;&gt;KYLIN-3672&lt;/a&gt;, and released in Kylin v2.5.2. Thanks to Shaofeng Shi of Kyligence Inc. for help.&lt;/p&gt;
+&lt;p&gt;参考&lt;a href=&quot;/docs/howto/howto_upgrade.html&quot;&gt;升级指南&lt;/a&gt;.&lt;/p&gt;
 
-&lt;p&gt;【1】&lt;a href=&quot;https://issues.apache.org/jira/browse/KYLIN-3672&quot;&gt;https://issues.apache.org/jira/browse/KYLIN-3672&lt;/a&gt;&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;反馈&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;&lt;em&gt;Author Zongwie Li as a Cisco engineer and a team member in the company’s Big Data architecture team, currently responsible for OLAP platform construction and customer business reporting systems.&lt;/em&gt;&lt;/p&gt;
+&lt;p&gt;如果您遇到问题或疑问,请发送邮件至 Apache Kylin dev 或 user 邮件列表:dev@kylin.apache.org,user@kylin.apache.org; 在发送之前,请确保您已通过发送电子邮件至 dev-subscribe@kylin.apache.org 或 user-subscribe@kylin.apache.org订阅了邮件列表。&lt;/p&gt;
 
+&lt;p&gt;&lt;em&gt;非常感谢所有贡献Apache Kylin的朋友!&lt;/em&gt;&lt;/p&gt;
 </description>
-        <pubDate>Thu, 17 Jan 2019 09:30:00 -0800</pubDate>
-        <link>http://kylin.apache.org/blog/2019/01/17/cisco-throughput-5x/</link>
-        <guid isPermaLink="true">http://kylin.apache.org/blog/2019/01/17/cisco-throughput-5x/</guid>
+        <pubDate>Fri, 18 Jan 2019 12:00:00 -0800</pubDate>
+        <link>http://kylin.apache.org/cn_blog/2019/01/18/release-v2.6.0/</link>
+        <guid isPermaLink="true">http://kylin.apache.org/cn_blog/2019/01/18/release-v2.6.0/</guid>
         
         
-        <category>blog</category>
+        <category>cn_blog</category>
         
       </item>
     

Added: kylin/site/images/blog/refine-query-cache/cache-signature.png
URL: http://svn.apache.org/viewvc/kylin/site/images/blog/refine-query-cache/cache-signature.png?rev=1864154&view=auto
==============================================================================
Binary file - no diff available.

Propchange: kylin/site/images/blog/refine-query-cache/cache-signature.png
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Added: kylin/site/images/blog/refine-query-cache/consistent-hashing.png
URL: http://svn.apache.org/viewvc/kylin/site/images/blog/refine-query-cache/consistent-hashing.png?rev=1864154&view=auto
==============================================================================
Binary file - no diff available.

Propchange: kylin/site/images/blog/refine-query-cache/consistent-hashing.png
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Added: kylin/site/images/blog/refine-query-cache/l1-l2-cache.png
URL: http://svn.apache.org/viewvc/kylin/site/images/blog/refine-query-cache/l1-l2-cache.png?rev=1864154&view=auto
==============================================================================
Binary file - no diff available.

Propchange: kylin/site/images/blog/refine-query-cache/l1-l2-cache.png
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream