You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kylin.apache.org by li...@apache.org on 2017/10/29 07:45:31 UTC
[1/2] kylin git commit: minor, fix how-to-release document
Repository: kylin
Updated Branches:
refs/heads/document e21b61cdb -> 52932cf0e
minor, fix how-to-release document
Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/42830770
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/42830770
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/42830770
Branch: refs/heads/document
Commit: 428307704e3409aea99007bf43ef31615dad2e1c
Parents: e21b61c
Author: lidongsjtu <li...@apache.org>
Authored: Sat Oct 28 22:53:48 2017 +0800
Committer: lidongsjtu <li...@apache.org>
Committed: Sun Oct 29 15:44:42 2017 +0800
----------------------------------------------------------------------
website/_dev/howto_release.md | 24 ++++++++++++------------
1 file changed, 12 insertions(+), 12 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/kylin/blob/42830770/website/_dev/howto_release.md
----------------------------------------------------------------------
diff --git a/website/_dev/howto_release.md b/website/_dev/howto_release.md
index 2807b0e..7fe387f 100644
--- a/website/_dev/howto_release.md
+++ b/website/_dev/howto_release.md
@@ -85,7 +85,7 @@ In the "servers" section, make sure the following servers be added, and replace
<username>#YOUR_APACHE_ID#</username>
<password>#YOUR_APACHE_PWD#</password>
</server>
-
+
<!-- To publish a website of some part of Maven -->
<server>
<id>apache.website</id>
@@ -97,7 +97,7 @@ In the "servers" section, make sure the following servers be added, and replace
<filePermissions>664</filePermissions>
<directoryPermissions>775</directoryPermissions>
</server>
-
+
<!-- To stage a website of some part of Maven -->
<server>
<id>stagingSite</id>
@@ -124,7 +124,7 @@ $ mvn clean
# Make sure all unit tests are passed
$ mvn test
-
+
# Check the `org.apache.kylin.common.KylinVersion` class, ensure the value of `CURRENT_KYLIN_VERSION` is the release version.
# Fix any license issues as reported by target/rat.txt
@@ -133,7 +133,7 @@ $ mvn -Papache-release -DskipTests -Dgpg.passphrase=${GPG_PASSPHRASE} install
Optionally, when the dry-run has succeeded, change install to deploy:
{% highlight bash %}
-$ mvn -Papache-release -Dgpg.passphrase=${GPG_PASSPHRASE} deploy
+$ mvn -Papache-release -DskipTests -Dgpg.passphrase=${GPG_PASSPHRASE} deploy
{% endhighlight %}
__Prepare and dry run a release__
@@ -157,7 +157,7 @@ $ git clean -xf
$ mvn clean
# Optionally, do a dry run of the release:prepare step, which sets version numbers.
-$ mvn -DdryRun=true -DskipTests -DreleaseVersion=X.Y.Z -DdevelopmentVersion=(X.Y.Z+1)-SNAPSHOT -Papache-release -Darguments="-Dgpg.passphrase=${GPG_PASSPHRASE}" release:prepare 2>&1 | tee /tmp/prepare-dry.log
+$ mvn -DdryRun=true -DskipTests -DreleaseVersion=X.Y.Z -DdevelopmentVersion=(X.Y.Z+1)-SNAPSHOT -Papache-release -Darguments="-Dgpg.passphrase=${GPG_PASSPHRASE} -DskipTests" release:prepare 2>&1 | tee /tmp/prepare-dry.log
{% endhighlight %}
__Check the dry run output:__
@@ -424,13 +424,13 @@ After publish the release, you need generate the binary packages and then put th
* Git checkout the tag for current release;
* Make a binary package by refering to [this doc](howto_package.html);
* Sign the generated binary package with gpg, e.g,:
-{% highlight bash %}
-gpg --armor --output apache-kylin-1.5.0-bin.tar.gz.asc --detach-sig apache-kylin-1.5.0-bin.tar.gz
-{% endhighlight %}
+ {% highlight bash %}
+ gpg --armor --output apache-kylin-1.5.0-bin.tar.gz.asc --detach-sig apache-kylin-1.5.0-bin.tar.gz
+ {% endhighlight %}
* Generate the md5 file for the binary package, e.g,:
-{% highlight bash %}
-md5sum < apache-kylin-1.5.0-bin.tar.gz > apache-kylin-1.5.0-bin.tar.gz.md5
-{% endhighlight %}
+ {% highlight bash %}
+ md5sum < apache-kylin-1.5.0-bin.tar.gz > apache-kylin-1.5.0-bin.tar.gz.md5
+ {% endhighlight %}
* Push the binary package, the signature file and the md5 file to the svn __dev__ repo, then run `svn mv <files-in-dev> <files-in-release>` to move them to svn __release__ repo.
* For different Hadoop/HBase version, you may need repeat the above steps;
* Add the files and then commit the svn changes.
@@ -450,7 +450,7 @@ Send one mail with subject like "[Announce] Apache Kylin x.y.z released" to foll
* Apache Kylin Dev mailing list: dev@kylin.apache.org
* Apache Kylin User mailing list: user@kylin.apache.org
* Apache Announce mailing list: announce@apache.org
-Please notice to always use your Apache mail address to send this;
+ Please notice to always use your Apache mail address to send this;
Here is a sample of announcement email (by studying Kafka's):
[2/2] kylin git commit: minor, remove beta for spark cubing
Posted by li...@apache.org.
minor, remove beta for spark cubing
Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/52932cf0
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/52932cf0
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/52932cf0
Branch: refs/heads/document
Commit: 52932cf0efd395c256a2cdecdfeb12b9320eaf88
Parents: 4283077
Author: lidongsjtu <li...@apache.org>
Authored: Sun Oct 29 15:44:57 2017 +0800
Committer: lidongsjtu <li...@apache.org>
Committed: Sun Oct 29 15:44:57 2017 +0800
----------------------------------------------------------------------
website/_docs21/index.md | 8 ++++++--
website/_docs21/tutorial/cube_spark.md | 8 ++++----
2 files changed, 10 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/kylin/blob/52932cf0/website/_docs21/index.md
----------------------------------------------------------------------
diff --git a/website/_docs21/index.md b/website/_docs21/index.md
index 12bf1b1..5515cac 100644
--- a/website/_docs21/index.md
+++ b/website/_docs21/index.md
@@ -5,6 +5,7 @@ categories: docs
permalink: /docs21/index.html
---
+
Welcome to Apache Kylin™: Extreme OLAP Engine for Big Data
------------
@@ -16,6 +17,7 @@ Document of prior versions:
* [v1.6.x document](/docs16/)
* [v1.5.x document](/docs15/)
+
Installation & Setup
------------
1. [Hadoop Env](install/hadoop_env.html)
@@ -25,15 +27,16 @@ Installation & Setup
5. [Run Kylin with Docker](install/kylin_docker.html)
+
Tutorial
------------
1. [Quick Start with Sample Cube](tutorial/kylin_sample.html)
2. [Web Interface](tutorial/web.html)
3. [Cube Wizard](tutorial/create_cube.html)
-3. [Cube Build and Job Monitoring](tutorial/cube_build_job.html)
+4. [Cube Build and Job Monitoring](tutorial/cube_build_job.html)
5. [SQL reference: by Apache Calcite](http://calcite.apache.org/docs/reference.html)
6. [Build Cube with Streaming Data](tutorial/cube_streaming.html)
-7. [Build Cube with Spark Engine (beta)](tutorial/cube_spark.html)
+7. [Build Cube with Spark Engine](tutorial/cube_spark.html)
8. [Cube Build Tuning](tutorial/cube_build_performance.html)
9. [Enable Query Pushdown](tutorial/query_pushdown.html)
@@ -53,6 +56,7 @@ Connectivity and APIs
10. [Connect from Apache Flink](tutorial/flink.html)
11. [Connect from Apache Hue](tutorial/hue.html)
+
Operations
------------
1. [Backup/restore Kylin metadata](howto/howto_backup_metadata.html)
http://git-wip-us.apache.org/repos/asf/kylin/blob/52932cf0/website/_docs21/tutorial/cube_spark.md
----------------------------------------------------------------------
diff --git a/website/_docs21/tutorial/cube_spark.md b/website/_docs21/tutorial/cube_spark.md
index 9af44cd..5400309 100644
--- a/website/_docs21/tutorial/cube_spark.md
+++ b/website/_docs21/tutorial/cube_spark.md
@@ -1,6 +1,6 @@
---
layout: docs21
-title: Build Cube with Spark (beta)
+title: Build Cube with Spark
categories: tutorial
permalink: /docs21/tutorial/cube_spark.html
---
@@ -10,7 +10,7 @@ Kylin v2.0 introduces the Spark cube engine, it uses Apache Spark to replace Map
## Preparation
To finish this tutorial, you need a Hadoop environment which has Kylin v2.1.0 or above installed. Here we will use Hortonworks HDP 2.4 Sandbox VM, the Hadoop components as well as Hive/HBase has already been started.
-## Install Kylin v2.1.0
+## Install Kylin v2.1.0 or above
Download the Kylin v2.1.0 for HBase 1.x from Kylin's download page, and then uncompress the tar ball into */usr/local/* folder:
@@ -104,7 +104,7 @@ $KYLIN_HOME/bin/kylin.sh start
{% endhighlight %}
-After Kylin is started, access Kylin web, edit the "kylin_sales" cube, in the "Advanced Setting" page, change the "Cube Engine" from "MapReduce" to "Spark (Beta)":
+After Kylin is started, access Kylin web, edit the "kylin_sales" cube, in the "Advanced Setting" page, change the "Cube Engine" from "MapReduce" to "Spark":
@@ -166,4 +166,4 @@ Click a specific job, there you will see the detail runtime information, that is
If you're a Kylin administrator but new to Spark, suggest you go through [Spark documents](https://spark.apache.org/docs/2.1.0/), and don't forget to update the configurations accordingly. You can enable Spark [Dynamic Resource Allocation](https://spark.apache.org/docs/2.1.0/job-scheduling.html#dynamic-resource-allocation) so that it can auto scale/shrink for different work load. Spark's performance relies on Cluster's memory and CPU resource, while Kylin's Cube build is a heavy task when having a complex data model and a huge dataset to build at one time. If your cluster resource couldn't fulfill, errors like "OutOfMemorry" will be thrown in Spark executors, so please use it properly. For Cube which has UHC dimension, many combinations (e.g, a full cube with more than 12 dimensions), or memory hungry measures (Count Distinct, Top-N), suggest to use the MapReduce engine. If your Cube model is simple, all measures are SUM/MIN/MAX/COUNT, source data is small to medium scale, Spark eng
ine would be a good choice. Besides, Streaming build isn't supported in this engine so far (KYLIN-2484).
-Now the Spark engine is in public beta; If you have any question, comment, or bug fix, welcome to discuss in dev@kylin.apache.org.
+If you have any question, comment, or bug fix, welcome to discuss in dev@kylin.apache.org.