You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by GitBox <gi...@apache.org> on 2018/11/01 13:47:08 UTC
[GitHub] shaofengshi closed pull request #323: Update FAQ, update storage and metadata cleanup doc, add the online v…
shaofengshi closed pull request #323: Update FAQ, update storage and metadata cleanup doc, add the online v…
URL: https://github.com/apache/kylin/pull/323
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/website/_docs/gettingstarted/best_practices.md b/website/_docs/gettingstarted/best_practices.md
index 191530e355..118a3889b0 100644
--- a/website/_docs/gettingstarted/best_practices.md
+++ b/website/_docs/gettingstarted/best_practices.md
@@ -24,4 +24,19 @@ List of articles about Kylin best practices contributed by community. Some of th
* [Kylin and Qlik Integration](https://github.com/albertoRamon/Kylin/tree/master/KylinWithQlik) (Thanks to [Ramón Portolés, Alberto](https://www.linkedin.com/in/alberto-ramon-portoles-a02b523b))
-* [How to use Hue with Kylin](https://github.com/albertoRamon/Kylin/tree/master/KylinWithHue) (Thanks to [Ramón Portolés, Alberto](https://www.linkedin.com/in/alberto-ramon-portoles-a02b523b))
\ No newline at end of file
+* [How to use Hue with Kylin](https://github.com/albertoRamon/Kylin/tree/master/KylinWithHue) (Thanks to [Ramón Portolés, Alberto](https://www.linkedin.com/in/alberto-ramon-portoles-a02b523b))
+
+
+**Here are online tutorials for self-studying Kylin:**
+
+- Free Kylin tutorial (registration needed), from the core developers, in Chinese:
+[https://www.chinahadoop.cn/search?q=kylin](https://www.chinahadoop.cn/search?q=kylin)
+
+- A paid Kylin tutorial on Udemy, in English:
+[https://www.udemy.com/apache-kylin-implementing-olap-on-the-hadoop-platform](https://www.udemy.com/apache-kylin-implementing-olap-on-the-hadoop-platform)
+
+- Tutorial of Kylin on Tableau, in Spanish:
+[https://www.youtube.com/watch?v=k6fBw8yA1NI](https://www.youtube.com/watch?v=k6fBw8yA1NI)
+
+Besides, there is also some video of Kylin on the conferences; You can search for them on Youtube:
+[https://www.youtube.com/results?search_query=Apache+Kylin](https://www.youtube.com/results?search_query=Apache+Kylin)
diff --git a/website/_docs/gettingstarted/faq.md b/website/_docs/gettingstarted/faq.md
index def985146e..0540857863 100644
--- a/website/_docs/gettingstarted/faq.md
+++ b/website/_docs/gettingstarted/faq.md
@@ -135,6 +135,10 @@ When the new cube is built, please disable or drop the old one.
If you can accept the absence of new dimensions for historical data, you can build the new cube since the end time of the old cube. And then create a hybrid model over the old and new cube.
+#### How to solve the data security problem of Tableau connection client?
+
+ * Kylin's ACL control can solve this problem. Different analysts have the authority to work on different projects for Kylin. When you create a Kylin ODBC DSN, you can map different links to different analyst accounts.
+
#### The query result is not exactly matched with that in Hive, what's the possible reason?
* Possible reasons:
diff --git a/website/_docs/howto/howto_backup_metadata.cn.md b/website/_docs/howto/howto_backup_metadata.cn.md
index 66ab3ed391..56db4c8b08 100644
--- a/website/_docs/howto/howto_backup_metadata.cn.md
+++ b/website/_docs/howto/howto_backup_metadata.cn.md
@@ -73,14 +73,14 @@ cd $KYLIN_HOME
## 从 metadata store 清理无用的资源
随着运行时间增长，类似字典、表快照的资源变得没有用（cube segment被丢弃或者合并了），但是它们依旧占用空间，你可以运行命令来找到并清除它们：
-首先，运行一个检查，这是安全的因为它不会改变任何东西：
+首先，运行一个检查，这是安全的因为它不会改变任何东西，通过添加 "--jobThreshold 30(默认值，您可以改为任何数字)" 参数，您可以设置要保留的 metadata resource 天数：
{% highlight Groff markup %}
-./bin/metastore.sh clean
+./bin/metastore.sh clean --jobThreshold 30
{% endhighlight %}
将要被删除的资源会被列出来：
接下来，增加 “--delete true” 参数来清理这些资源；在这之前，你应该确保已经备份 metadata store：
{% highlight Groff markup %}
-./bin/metastore.sh clean --delete true
+./bin/metastore.sh clean --delete true --jobThreshold 30
{% endhighlight %}
diff --git a/website/_docs/howto/howto_backup_metadata.md b/website/_docs/howto/howto_backup_metadata.md
index bbc0d0f5ff..deaaa70147 100644
--- a/website/_docs/howto/howto_backup_metadata.md
+++ b/website/_docs/howto/howto_backup_metadata.md
@@ -74,14 +74,14 @@ When developing/debugging Kylin, typically you have a dev machine with an IDE, a
## Cleanup unused resources from metadata store
As time goes on, some resources like dictionary, table snapshots became useless (as the cube segment be dropped or merged), but they still take space there; You can run command to find and cleanup them from metadata store:
-Firstly, run a check, this is safe as it will not change anything:
+Firstly, run a check, this is safe as it will not change anything, you can set the number of days to keep metadata resource by adding the "--jobThreshold 30(default, you can change to any number)" option:
{% highlight Groff markup %}
-./bin/metastore.sh clean
+./bin/metastore.sh clean --jobThreshold 30
{% endhighlight %}
The resources that will be dropped will be listed;
Next, add the "--delete true" parameter to cleanup those resources; before this, make sure you have made a backup of the metadata store;
{% highlight Groff markup %}
-./bin/metastore.sh clean --delete true
+./bin/metastore.sh clean --delete true --jobThreshold 30
{% endhighlight %}
diff --git a/website/_docs/howto/howto_cleanup_storage.cn.md b/website/_docs/howto/howto_cleanup_storage.cn.md
index b543fd7aa2..2dbd1d63d8 100644
--- a/website/_docs/howto/howto_cleanup_storage.cn.md
+++ b/website/_docs/howto/howto_cleanup_storage.cn.md
@@ -5,7 +5,7 @@ categories: 帮助
permalink: /cn/docs/howto/howto_cleanup_storage.html
---
-Kylin在构建cube期间会在HDFS上生成中间文件；除此之外，当清理/删除/合并cube时，一些HBase表可能被遗留在HBase却以后再也不会被查询；虽然Kylin已经开始做自动化的垃圾回收，但不一定能覆盖到所有的情况；你可以定期做离线的存储清理：
+Kylin 在构建 cube 期间会在 HDFS 上生成中间文件；除此之外，当清理/删除/合并 cube 时，一些 HBase 表可能被遗留在 HBase 却以后再也不会被查询；虽然 Kylin 已经开始做自动化的垃圾回收，但不一定能覆盖到所有的情况；你可以定期做离线的存储清理：
步骤：
1. 检查哪些资源可以清理，这一步不会删除任何东西：
@@ -13,9 +13,14 @@ Kylin在构建cube期间会在HDFS上生成中间文件；除此之外，当清
export KYLIN_HOME=/path/to/kylin_home
${KYLIN_HOME}/bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete false
{% endhighlight %}
-请将这里的 (version) 替换为你安装的Kylin jar版本。
+请将这里的 (version) 替换为你安装的 Kylin jar 版本。
2. 你可以抽查一两个资源来检查它们是否已经没有被引用了；然后加上“--delete true”选项进行清理。
{% highlight Groff markup %}
${KYLIN_HOME}/bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete true
{% endhighlight %}
-完成后，中间HDFS上的中间文件和HTable会被移除。
+完成后，Hive 里的中间表, HDFS 上的中间文件及 HBase 中的 HTables 都会被移除。
+3. 如果您想要删除所有资源；可添加 "--force true" 选项：
+{% highlight Groff markup %}
+${KYLIN_HOME}/bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --force true --delete true
+{% endhighlight %}
+完成后，Hive 中所有的中间表, HDFS 上所有的中间文件及 HBase 中的 HTables 都会被移除。
diff --git a/website/_docs/howto/howto_cleanup_storage.md b/website/_docs/howto/howto_cleanup_storage.md
index f796bb3585..043116a223 100644
--- a/website/_docs/howto/howto_cleanup_storage.md
+++ b/website/_docs/howto/howto_cleanup_storage.md
@@ -19,4 +19,9 @@ Here please replace (version) with the specific Kylin jar version in your instal
{% highlight Groff markup %}
${KYLIN_HOME}/bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete true
{% endhighlight %}
-On finish, the intermediate HDFS location and HTables should be dropped;
+On finish, the intermediate Hive tables, HDFS location and HTables should be dropped;
+3. If you want to delete all resources, then add the "--force true" option to start the cleanup:
+{% highlight Groff markup %}
+${KYLIN_HOME}/bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --force true --delete true
+{% endhighlight %}
+On finish, all the intermediate Hive tables, HDFS location and HTables should be dropped;
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services