You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kylin.apache.org by li...@apache.org on 2016/02/13 13:23:35 UTC
svn commit: r1730196 - in /kylin/site: development/ images/develop/
Author: liyang
Date: Sat Feb 13 12:23:35 2016
New Revision: 1730196
URL: http://svn.apache.org/viewvc?rev=1730196&view=rev
Log:
plugin arch doc
Added:
kylin/site/development/plugin_arch.html
kylin/site/images/develop/plugin_arch_adaptor_pattern.png (with props)
kylin/site/images/develop/plugin_arch_factory_pattern.png (with props)
kylin/site/images/develop/plugin_arch_overview.png (with props)
Modified:
kylin/site/development/about_temp_files.html
kylin/site/development/dev_env.html
kylin/site/development/howto_contribute.html
kylin/site/development/howto_docs.html
kylin/site/development/howto_package.html
kylin/site/development/howto_release.html
kylin/site/development/howto_test.html
kylin/site/development/index.html
kylin/site/development/new_metadata.html
kylin/site/development/web_tech.html
Modified: kylin/site/development/about_temp_files.html
URL: http://svn.apache.org/viewvc/kylin/site/development/about_temp_files.html?rev=1730196&r1=1730195&r2=1730196&view=diff
==============================================================================
--- kylin/site/development/about_temp_files.html (original)
+++ kylin/site/development/about_temp_files.html Sat Feb 13 12:23:35 2016
@@ -372,6 +372,39 @@
+
+
+ <li><a href="/development/plugin_arch.html" class="list-group-item-lay pjaxlink" id="navlist">Plugin Architecture</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
<li><a href="/development/new_metadata.html" class="list-group-item-lay pjaxlink" id="navlist">New Metadata Model</a></li>
@@ -388,6 +421,8 @@
+
+
Modified: kylin/site/development/dev_env.html
URL: http://svn.apache.org/viewvc/kylin/site/development/dev_env.html?rev=1730196&r1=1730195&r2=1730196&view=diff
==============================================================================
--- kylin/site/development/dev_env.html (original)
+++ kylin/site/development/dev_env.html Sat Feb 13 12:23:35 2016
@@ -372,6 +372,39 @@
+
+
+ <li><a href="/development/plugin_arch.html" class="list-group-item-lay pjaxlink" id="navlist">Plugin Architecture</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
<li><a href="/development/new_metadata.html" class="list-group-item-lay pjaxlink" id="navlist">New Metadata Model</a></li>
@@ -388,6 +421,8 @@
+
+
Modified: kylin/site/development/howto_contribute.html
URL: http://svn.apache.org/viewvc/kylin/site/development/howto_contribute.html?rev=1730196&r1=1730195&r2=1730196&view=diff
==============================================================================
--- kylin/site/development/howto_contribute.html (original)
+++ kylin/site/development/howto_contribute.html Sat Feb 13 12:23:35 2016
@@ -372,6 +372,39 @@
+
+
+ <li><a href="/development/plugin_arch.html" class="list-group-item-lay pjaxlink" id="navlist">Plugin Architecture</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
<li><a href="/development/new_metadata.html" class="list-group-item-lay pjaxlink" id="navlist">New Metadata Model</a></li>
@@ -388,6 +421,8 @@
+
+
Modified: kylin/site/development/howto_docs.html
URL: http://svn.apache.org/viewvc/kylin/site/development/howto_docs.html?rev=1730196&r1=1730195&r2=1730196&view=diff
==============================================================================
--- kylin/site/development/howto_docs.html (original)
+++ kylin/site/development/howto_docs.html Sat Feb 13 12:23:35 2016
@@ -372,6 +372,39 @@
+
+
+ <li><a href="/development/plugin_arch.html" class="list-group-item-lay pjaxlink" id="navlist">Plugin Architecture</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
<li><a href="/development/new_metadata.html" class="list-group-item-lay pjaxlink" id="navlist">New Metadata Model</a></li>
@@ -388,6 +421,8 @@
+
+
Modified: kylin/site/development/howto_package.html
URL: http://svn.apache.org/viewvc/kylin/site/development/howto_package.html?rev=1730196&r1=1730195&r2=1730196&view=diff
==============================================================================
--- kylin/site/development/howto_package.html (original)
+++ kylin/site/development/howto_package.html Sat Feb 13 12:23:35 2016
@@ -372,6 +372,39 @@
+
+
+ <li><a href="/development/plugin_arch.html" class="list-group-item-lay pjaxlink" id="navlist">Plugin Architecture</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
<li><a href="/development/new_metadata.html" class="list-group-item-lay pjaxlink" id="navlist">New Metadata Model</a></li>
@@ -388,6 +421,8 @@
+
+
Modified: kylin/site/development/howto_release.html
URL: http://svn.apache.org/viewvc/kylin/site/development/howto_release.html?rev=1730196&r1=1730195&r2=1730196&view=diff
==============================================================================
--- kylin/site/development/howto_release.html (original)
+++ kylin/site/development/howto_release.html Sat Feb 13 12:23:35 2016
@@ -372,6 +372,39 @@
+
+
+ <li><a href="/development/plugin_arch.html" class="list-group-item-lay pjaxlink" id="navlist">Plugin Architecture</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
<li><a href="/development/new_metadata.html" class="list-group-item-lay pjaxlink" id="navlist">New Metadata Model</a></li>
@@ -388,6 +421,8 @@
+
+
Modified: kylin/site/development/howto_test.html
URL: http://svn.apache.org/viewvc/kylin/site/development/howto_test.html?rev=1730196&r1=1730195&r2=1730196&view=diff
==============================================================================
--- kylin/site/development/howto_test.html (original)
+++ kylin/site/development/howto_test.html Sat Feb 13 12:23:35 2016
@@ -372,6 +372,39 @@
+
+
+ <li><a href="/development/plugin_arch.html" class="list-group-item-lay pjaxlink" id="navlist">Plugin Architecture</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
<li><a href="/development/new_metadata.html" class="list-group-item-lay pjaxlink" id="navlist">New Metadata Model</a></li>
@@ -405,6 +438,8 @@
+
+
<li><a href="/development/web_tech.html" class="list-group-item-lay pjaxlink" id="navlist">Kylin Web Summary</a></li>
@@ -441,7 +476,7 @@
<article class="post-content" >
<p>In general, there should be unit tests to cover individual classes; there must be integration test to cover end-to-end scenarios like build, merge, and query. Unit test must run independently (does not require an external sandbox).</p>
-<h2 id="test-2x-branches">Test 2.x branches</h2>
+<h2 id="x-branches">2.x branches</h2>
<ul>
<li><code class="highlighter-rouge">mvn test</code> to run unit tests, which has a limited test coverage.
@@ -453,15 +488,15 @@
</li>
<li><code class="highlighter-rouge">dev-support/test_all_against_hdp_2_2_4_2_2.sh</code> to run integration tests, which has the best test coverage.
<ul>
- <li>Integration tests <strong>better be run on a Hadoop sandbox</strong>. We suggest to checkout a copy of code in your sandbox and direct run the test_all_against_hdp_2_2_4_2_2.sh in it. If you donât want to put codes on sandbox, refer to <strong>More on 2.x UT/IT separation</strong></li>
+ <li>Integration tests <strong>must run on a Hadoop sandbox</strong>. Make sure all changes you want to test are avaiable on sandbox.</li>
<li>As the name indicates, the script is only for hdp 2.2.4.2, but you get the idea of how integration test run from it.</li>
<li>The integration tests start from generate random data, then build cube, merge cube, and finally query the result and compare to H2 DB.</li>
- <li>The integration tests take one to two hours to complete.</li>
+ <li>The integration tests take a few hours to complete.</li>
</ul>
</li>
</ul>
-<h2 id="test-1x-branches">Test 1.x branches</h2>
+<h2 id="x-branches-1">1.x branches</h2>
<ul>
<li><code class="highlighter-rouge">mvn test</code> to run unit tests, which has a limited test coverage.
@@ -480,32 +515,6 @@
</li>
</ul>
-<h2 id="more-on-2x-utit-separation">More on 2.x UT/IT separation</h2>
-
-<p>From Kylin 2.0 you can run UT(Unit test), environment cube provision and IT(Integration test) separately. <br />
-Running <code class="highlighter-rouge">mvn verify -Dhdp.version=2.2.4.2-2</code> (assume youâre on your sandbox) is all you need to run a complete all the test suites.</p>
-
-<p>It will execute the following steps sequentially:</p>
-
-<div class="highlighter-rouge"><pre class="highlight"><code>1. Build Artifacts
-2. Run all UTs (takes few minutes)
-3. Provision cubes on the sandbox environment for IT uasge (takes 1~2 hours)
-4. Run all ITs (takes few tens of minutes)
-5. verify jar stuff
-</code></pre>
-</div>
-
-<p>If your code change is minor and it merely requires running UT, use: <br />
-<code class="highlighter-rouge">mvn test</code><br />
-If your sandbox is already provisioned and your code change will not affect the result of sandbox provision, (and you donât want to wait hours of provision) just run the following commands to separately run UT and IT: <br />
-<code class="highlighter-rouge">mvn test</code><br />
-<code class="highlighter-rouge">mvn failsafe:integration-test</code></p>
-
-<h3 id="cube-provision">Cube Provision</h3>
-
-<p>Environment cube provision is indeed running kylin cubing jobs to prepare example cubes in the sandbox. These prepared cubes will be used by the ITs. Currently provision step is bound with the maven pre-integration-test phase, and it contains running BuildCubeWithEngine (HBase required), BuildCubeWithStream(Kafka required) and BuildIIWithStream(Kafka Required). You can run the mvn commands on you sandbox or your develop computer. For the latter case you need to set kylin.job.run.as.remote.cmd=true in <strong>$KYLIN_HOME/examples/test_case_data/sandbox/kylin.properties</strong>. <br />
-Try appending <code class="highlighter-rouge">-DfastBuildMode=true</code> to mvn verify command to speed up provision by skipping incremental cubing.</p>
-
<h2 id="more-on-1x-mini-cluster">More on 1.x Mini Cluster</h2>
<p>Kylin 1.x used to move as many as possible unit test cases from sandbox to HBase mini cluster (not any more in 2.x), so that user can run tests easily in local without a hadoop sandbox. Two maven profiles are created in the root pom.xml, âdefaultâ and âsandboxâ. The default profile will startup a HBase Mini Cluster to prepare the test data and run the unit tests (the test cases that are not supported by Mini cluster have been added in the âexcludeâ list). If you want to keep using Sandbox to run test, just run <code class="highlighter-rouge">mvn test -P sandbox</code></p>
@@ -520,7 +529,6 @@ Try appending <code class="highlighter-r
</ul>
<h3 id="to-ensure-mini-cluster-can-run-successfully-you-need">To ensure Mini cluster can run successfully, you need</h3>
-
<ul>
<li>Make sure JAVA_HOME is properly set</li>
</ul>
Modified: kylin/site/development/index.html
URL: http://svn.apache.org/viewvc/kylin/site/development/index.html?rev=1730196&r1=1730195&r2=1730196&view=diff
==============================================================================
--- kylin/site/development/index.html (original)
+++ kylin/site/development/index.html Sat Feb 13 12:23:35 2016
@@ -372,6 +372,39 @@
+
+
+ <li><a href="/development/plugin_arch.html" class="list-group-item-lay pjaxlink" id="navlist">Plugin Architecture</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
<li><a href="/development/new_metadata.html" class="list-group-item-lay pjaxlink" id="navlist">New Metadata Model</a></li>
@@ -388,6 +421,8 @@
+
+
Modified: kylin/site/development/new_metadata.html
URL: http://svn.apache.org/viewvc/kylin/site/development/new_metadata.html?rev=1730196&r1=1730195&r2=1730196&view=diff
==============================================================================
--- kylin/site/development/new_metadata.html (original)
+++ kylin/site/development/new_metadata.html Sat Feb 13 12:23:35 2016
@@ -372,6 +372,39 @@
+
+
+ <li><a href="/development/plugin_arch.html" class="list-group-item-lay pjaxlink" id="navlist">Plugin Architecture</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
<li><a href="/development/new_metadata.html" class="list-group-item-lay pjaxlink" id="navlist">New Metadata Model</a></li>
@@ -388,6 +421,8 @@
+
+
Added: kylin/site/development/plugin_arch.html
URL: http://svn.apache.org/viewvc/kylin/site/development/plugin_arch.html?rev=1730196&view=auto
==============================================================================
--- kylin/site/development/plugin_arch.html (added)
+++ kylin/site/development/plugin_arch.html Sat Feb 13 12:23:35 2016
@@ -0,0 +1,579 @@
+<!--
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+-->
+
+<!doctype html>
+<html>
+ <!--
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+-->
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>Apache Kylin | Plugin Architecture</title>
+ <meta name="description" content="Apache Kylin Home">
+ <meta name="author" content="Apache Kylin">
+ <link rel="shortcut icon" href="fav.png" type="image/png">
+
+
+
+<link rel="stylesheet" href="/assets/css/animate.css">
+<!-- Bootstrap -->
+<link rel="stylesheet" href="/assets/css/bootstrap.min.css">
+
+<!-- Fonts -->
+<!-- <link rel="stylesheet" href="http://fonts.googleapis.com/css?family=Alice|Open+Sans:400,300,700"> -->
+
+<!-- Icons -->
+<link rel="stylesheet" href="/assets/css/font-awesome.min.css">
+
+ <!-- Custom styles -->
+ <link rel="stylesheet" href="/assets/css/styles.css">
+ <link rel="stylesheet" href="/assets/css/docs.css">
+ <link rel="stylesheet" href="/assets/css/pygments.css">
+
+ <link rel="canonical" href="http://kylin.apache.org/development/plugin_arch.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Kylin" href="http://kylin.apache.org/feed.xml" />
+
+<!--[if lt IE 9]> <script src="assets/js/html5shiv.js"></script> <![endif]-->
+<script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ //oringal tracker for kylin.io
+ ga('create', 'UA-55534813-1', 'auto');
+ //new tracker for kylin.apache.org
+ ga('create', 'UA-55534813-2', 'auto', {'name':'toplevel'});
+
+ ga('send', 'pageview');
+ ga('toplevel.send', 'pageview');
+
+
+</script>
+<script type="text/javascript" src="/assets/js/jquery-1.9.1.min.js"></script>
+<script type="text/javascript" src="/assets/js/nside.js"></script> </script>
+<script type="text/javascript" src="/assets/js/nnav.js"></script> </script>
+</head>
+
+ <body>
+ <!--
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+-->
+
+<header id="header" >
+
+ <div id="head" class="parallax" parallax-speed="3" >
+ <div id="logo" class="text-center"> <img class="img-circle" id="circlelogo" src="/assets/images/kylin_logo.jpg"> <span class="title" >Apache Kylinâ¢</span> <span class="tagline">Extreme OLAP Engine for Big Data</span>
+ </div>
+ </div>
+
+
+ <!-- Main Menu -->
+ <nav class="navbar navbar-default" role="navigation" id="nav-wrapper">
+ <div class="container-fluid" id="nav">
+ <!--
+ <img class="img-circle" width="40px" height="40px" id="circlelogo" src="/assets/images/kylin_logo.jpg">
+ -->
+ <!-- Brand and toggle get grouped for better mobile display -->
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+
+ </div>
+
+ <!-- Collect the nav links, forms, and other content for toggling -->
+ <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
+ <ul class="nav navbar-nav">
+ <li><a href="/">Home</a></li>
+ <li><a href="/docs" >Docs</a></li>
+ <li><a href="/download">Download</li>
+ <li><a href="/community" >Community</a></li>
+ <li><a href="/development" >Development</a></li>
+ <li><a href="/blog">Blog</li>
+ <li><a href="/cn" >ä¸æç</a></li>
+ <li><a href="https://twitter.com/apachekylin" target="_blank" class="fa fa-twitter fa-lg" title="Twitter: @ApacheKylin" ></a></li>
+ <li><a href="https://github.com/apache/kylin" target="_blank" class="fa fa-github-alt fa-lg" title="Github: apache/kylin" ></a></li>
+ <li><a href="https://www.facebook.com/kylinio" target="_blank" class="fa fa-facebook fa-lg" title="Facebook: kylin.io" ></a></li>
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </div><!-- /.container-fluid -->
+</nav>
+ </header>
+
+
+ <div class="container">
+ <div class="row">
+ <!--
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+-->
+
+<div class="col-md-3 col-lg-3 col-xs-4 aside1 visible-md visible-lg" id="nside1" style=" padding-top: 2em">
+ <ul class="nav nav-pills nav-stacked">
+
+ <li><a href="#titleDevelopment Guide" data-toggle="collapse" id="navtitle">Development Guide</a></li>
+ <div class="collapse in">
+ <div class="list-group" id="list1">
+ <ul style="list-style-type:disc">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li><a href="/development/index.html" class="list-group-item-lay pjaxlink" id="navlist">Development Quick Guide</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li><a href="/development/dev_env.html" class="list-group-item-lay pjaxlink" id="navlist">Setup Development Env</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li><a href="/development/howto_test.html" class="list-group-item-lay pjaxlink" id="navlist">How to Test</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li><a href="/development/howto_contribute.html" class="list-group-item-lay pjaxlink" id="navlist">How to Contribute</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li><a href="/development/howto_docs.html" class="list-group-item-lay pjaxlink" id="navlist">How to Write Document</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li><a href="/development/howto_package.html" class="list-group-item-lay pjaxlink" id="navlist">How to Build Binary Package</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li><a href="/development/howto_release.html" class="list-group-item-lay pjaxlink" id="navlist">How to Making a Release</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li><a href="/development/plugin_arch.html" class="list-group-item-lay pjaxlink" id="navlist">Plugin Architecture</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li><a href="/development/new_metadata.html" class="list-group-item-lay pjaxlink" id="navlist">New Metadata Model</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li><a href="/development/web_tech.html" class="list-group-item-lay pjaxlink" id="navlist">Kylin Web Summary</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li><a href="/development/about_temp_files.html" class="list-group-item-lay pjaxlink" id="navlist">About Temp Files</a></li>
+
+
+
+
+
+
+ <ul>
+ </div>
+</div>
+
+
+ </ul>
+</div>
+ <div class="col-md-9 col-lg-9 col-xs-14 aside2">
+ <div id="container">
+ <div id="pjax">
+ <h1 class="post-title">Plugin Architecture</h1>
+ <article class="post-content" >
+ <p>The plugin architecture aims to make Kylin extensible regarding computation framework, data source, and cube storage. As of v1, Kylin tightly couples with Hadoop MapReduce as computation framework, Hive as data source, and HBase as storage. Questions came like: could Kylin use Spark as cube engine, or how about a different storage like Cassandra. We want to be open to different options, and to make sure Kylin evolve with the best tech stacks. That is why the plugin architecture is introduced in Kylin v2.</p>
+
+<p><img src="/images/develop/plugin_arch_overview.png" alt="Plugin Architecture Overview" /></p>
+
+<h2 id="how-it-works">How it Works</h2>
+
+<p>The cube metadata defines the type of engine, source, and storage that a cube depends on. Factory pattern is used to construct instances of each dependency. Adaptor pattern is used to connect the parts together.</p>
+
+<p>For example a cube descriptor may contains:</p>
+
+<ul>
+ <li>fact_table: <code class="highlighter-rouge">SOME_HIVE_TABLE</code></li>
+ <li>engine_type: <code class="highlighter-rouge">2</code> (MR Engine v2)</li>
+ <li>storage_type: <code class="highlighter-rouge">2</code> (HBase Storage v2)</li>
+</ul>
+
+<p>Based on the metadata, factories creates MR engine, Hive data source, and HBase storage.</p>
+
+<p><img src="/images/develop/plugin_arch_factory_pattern.png" alt="Plugin Architecture Factory Pattern" /></p>
+
+<p>The engine is like a motherboard, on which source and storage must be plugged as defined by the IN and OUT interfaces. Data source and storage must adapt to the interfaces in order to be connected to engine motherboard.</p>
+
+<p><img src="/images/develop/plugin_arch_adaptor_pattern.png" alt="Plugin Architecture Adaptor Pattern" /></p>
+
+<p>Once the above object graph is created and connected, engine can drive the cube build process.</p>
+
+<h2 id="the-benefits-of-plugin-architecture">The Benefits of Plugin Architecture</h2>
+
+<ul>
+ <li>Freedom
+ <ul>
+ <li>Zoo break, not bound to Hadoop any more</li>
+ <li>Free to go to a better engine or storage</li>
+ </ul>
+ </li>
+ <li>Extensibility
+ <ul>
+ <li>Accept any input, e.g. Kafka</li>
+ <li>Embrace next-gen distributed platform, e.g. Spark</li>
+ </ul>
+ </li>
+ <li>Flexibility
+ <ul>
+ <li>Choose different engine for different data set</li>
+ </ul>
+ </li>
+</ul>
+
+
+ </article>
+ </div>
+ </div>
+ </div>
+ </div>
+ </div>
+ <!--
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+-->
+
+<footer id="underfooter">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-12 widget">
+ <div class="widget-body" style="text-align:center">
+ <a href="http://www.apache.org">
+ <img id="asf-logo" alt="Apache Software Foundation" src="/assets/images/feather-small.gif">
+ </a>
+
+ <div>
+ The contents of this website are © 2015 Apache Software Foundation under the terms of the <a
+ href="http://www.apache.org/licenses/LICENSE-2.0"> Apache License v2 </a>. Apache Kylin and
+ its logo are trademarks of the Apache Software Foundation.
+ </div>
+
+ </div>
+ </div>
+ </div>
+ <!-- /row of widgets -->
+
+ </div>
+ <div></div>
+
+</footer>
+
+ <script src="/assets/js/jquery-1.9.1.min.js"></script>
+ <script src="/assets/js/bootstrap.min.js"></script>
+ <script src="/assets/js/main.js"></script>
+ </body>
+</html>
Modified: kylin/site/development/web_tech.html
URL: http://svn.apache.org/viewvc/kylin/site/development/web_tech.html?rev=1730196&r1=1730195&r2=1730196&view=diff
==============================================================================
--- kylin/site/development/web_tech.html (original)
+++ kylin/site/development/web_tech.html Sat Feb 13 12:23:35 2016
@@ -372,6 +372,39 @@
+
+
+ <li><a href="/development/plugin_arch.html" class="list-group-item-lay pjaxlink" id="navlist">Plugin Architecture</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
<li><a href="/development/new_metadata.html" class="list-group-item-lay pjaxlink" id="navlist">New Metadata Model</a></li>
@@ -388,6 +421,8 @@
+
+
Added: kylin/site/images/develop/plugin_arch_adaptor_pattern.png
URL: http://svn.apache.org/viewvc/kylin/site/images/develop/plugin_arch_adaptor_pattern.png?rev=1730196&view=auto
==============================================================================
Binary file - no diff available.
Propchange: kylin/site/images/develop/plugin_arch_adaptor_pattern.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: kylin/site/images/develop/plugin_arch_factory_pattern.png
URL: http://svn.apache.org/viewvc/kylin/site/images/develop/plugin_arch_factory_pattern.png?rev=1730196&view=auto
==============================================================================
Binary file - no diff available.
Propchange: kylin/site/images/develop/plugin_arch_factory_pattern.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: kylin/site/images/develop/plugin_arch_overview.png
URL: http://svn.apache.org/viewvc/kylin/site/images/develop/plugin_arch_overview.png?rev=1730196&view=auto
==============================================================================
Binary file - no diff available.
Propchange: kylin/site/images/develop/plugin_arch_overview.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream