You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kylin.apache.org by li...@apache.org on 2016/07/23 06:00:50 UTC
svn commit: r1753857 - in /kylin/site:
blog/2016/06/10/standalone-hbase-cluster/index.html feed.xml
Author: lidong
Date: Sat Jul 23 06:00:50 2016
New Revision: 1753857
URL: http://svn.apache.org/viewvc?rev=1753857&view=rev
Log:
update blog standalone-hbase-cluster with "Enable NN HA" section
Modified:
kylin/site/blog/2016/06/10/standalone-hbase-cluster/index.html
kylin/site/feed.xml
Modified: kylin/site/blog/2016/06/10/standalone-hbase-cluster/index.html
URL: http://svn.apache.org/viewvc/kylin/site/blog/2016/06/10/standalone-hbase-cluster/index.html?rev=1753857&r1=1753856&r2=1753857&view=diff
==============================================================================
--- kylin/site/blog/2016/06/10/standalone-hbase-cluster/index.html (original)
+++ kylin/site/blog/2016/06/10/standalone-hbase-cluster/index.html Sat Jul 23 06:00:50 2016
@@ -206,18 +206,33 @@
<p>Notice that the value should keep consistent with the Namenode address of <code class="highlighter-rouge">root.dir</code> on HBase Master node, to ensure bulkload into hbase.</p>
-<h2 id="using-nn-ha">Using NN HA</h2>
-<p>HDFS Namenode HA improved the availablity of cluster significantly, and maybe the HBase cluster enabled it. Apache Kylin doesn’t support the HA perfectly for now, and here’s the workaroud:</p>
+<h2 id="enable-nn-ha">Enable NN HA</h2>
+<p>HDFS Namenode HA improved the availablity of cluster significantly, and maybe the HBase cluster enabled it.<br />
+To enable NN HA on HBase cluster, set <code class="highlighter-rouge">kylin.hbase.cluster.fs</code> as NN-HA format path of HBase cluster in kylin.properties, like <code class="highlighter-rouge">hdfs://hbase-cluster:8020</code>.</p>
+
+<p>Since the MR Job of cube building access both main cluster and hbase cluster, exceptions may be threw during task running. The reason is task need to access HBase cluster with NN-HA related configs, which usually not deployed in the main cluster.</p>
+
+<p>These exceptions could be resolved by configs updating. In details, merge NN-HA related configs of two clusters, including <code class="highlighter-rouge">dfs.nameservices</code>, <code class="highlighter-rouge">dfs.ha.namenodes.</code>, <code class="highlighter-rouge">dfs.client.failover.proxy.provider.</code> and other related. These configs should be updated into hdfs-site.xml in Kylin Server and Resource Manager of main cluster, also be added into kylin_job_conf.xml.</p>
+
+<p>Howerver, it may be difficult to update configs in production environments. Here’s another way which avoing configs updating:</p>
<ul>
- <li>Add all <code class="highlighter-rouge">dfs.nameservices</code> related configs of HBase Cluster into <code class="highlighter-rouge">hadoop/etc/hadoop/hdfs-site.xml</code> in Kylin Server, to make sure that can access HBase Cluster using hdfs shell with nameservice path</li>
- <li>Add all <code class="highlighter-rouge">dfs.nameservices</code> related configs of both two clusters into <code class="highlighter-rouge">kylin_job_conf.xml</code>, to make sure that the MR job can access hbase cluster with nameservice path</li>
+ <li>Update HBase client in Kylin Server with patch HBASE-14347, and set <code class="highlighter-rouge">hbase.use.dynamic.jar=false</code> in hbase-site.xml</li>
+ <li>Update Kylin Server with patch KYLIN-1910, and set <code class="highlighter-rouge">kylin.hbase.cluster.hdfs.config.file=hbase.hdfs.xml</code> in kylin.properties. The <code class="highlighter-rouge">hbase.hdfs.xml</code> meaning the hdfs-site.xml of HBase cluster, put it in the same dir of kylin.properties</li>
+ <li>Update Hadoop of Kylin Server and Resource Manager with patch YARN-3021</li>
</ul>
<h2 id="troubleshooting">TroubleShooting</h2>
+
<ul>
<li>UnknownHostException occurs during Cube Building<br />
-It usually occurs with HBase HA nameservice config, please refer the above section “Using NN HA”</li>
+It usually occurs with HBase HA nameservice config, please refer the above section “Enable NN HA”</li>
+ <li>‘Error when open connection hbase’ during Kylin Server startup<br />
+HBase tries to mkdir tmp dirs on hdfs during connection setup, failed with lack of NN-HA related configs, refer “Enable NN HA”</li>
+ <li>Failed to submit cube building job<br />
+HBase cluster path will be parsed during generate cube building job, failed with lack of NN-HA related configs, refer “Enable NN HA”</li>
+ <li>Cube Building Step ‘Convert Cuboid Data to HFile’ failed<br />
+Usually occurred with Kerberos Authentication. Resource Manager tries to renew all tokens when job submitting, the renew of HBase cluster token will be failed, with lack of NN-HA related configs, refer “Enable NN HA”</li>
<li>HFile BulkLoading Stucks for long time<br />
Check the regionserver log, there should be lots of error log, with WrongFS exception. Make sure the namenode address in <code class="highlighter-rouge">kylin.properites/kylin.hbase.cluster.fs</code> and hbase master node <code class="highlighter-rouge">hbase-site.xml/root.dir</code> is same</li>
</ul>
Modified: kylin/site/feed.xml
URL: http://svn.apache.org/viewvc/kylin/site/feed.xml?rev=1753857&r1=1753856&r2=1753857&view=diff
==============================================================================
--- kylin/site/feed.xml (original)
+++ kylin/site/feed.xml Sat Jul 23 06:00:50 2016
@@ -19,8 +19,8 @@
<description>Apache Kylin Home</description>
<link>http://kylin.apache.org/</link>
<atom:link href="http://kylin.apache.org/feed.xml" rel="self" type="application/rss+xml"/>
- <pubDate>Fri, 01 Jul 2016 06:59:27 -0700</pubDate>
- <lastBuildDate>Fri, 01 Jul 2016 06:59:27 -0700</lastBuildDate>
+ <pubDate>Sat, 23 Jul 2016 06:59:37 -0700</pubDate>
+ <lastBuildDate>Sat, 23 Jul 2016 06:59:37 -0700</lastBuildDate>
<generator>Jekyll v2.5.3</generator>
<item>
@@ -74,18 +74,33 @@
<p>Notice that the value should keep consistent with the Namenode address of <code class="highlighter-rouge">root.dir</code> on HBase Master node, to ensure bulkload into hbase.</p>
-<h2 id="using-nn-ha">Using NN HA</h2>
-<p>HDFS Namenode HA improved the availablity of cluster significantly, and maybe the HBase cluster enabled it. Apache Kylin doesn’t support the HA perfectly for now, and here’s the workaroud:</p>
+<h2 id="enable-nn-ha">Enable NN HA</h2>
+<p>HDFS Namenode HA improved the availablity of cluster significantly, and maybe the HBase cluster enabled it.<br />
+To enable NN HA on HBase cluster, set <code class="highlighter-rouge">kylin.hbase.cluster.fs</code> as NN-HA format path of HBase cluster in kylin.properties, like <code class="highlighter-rouge">hdfs://hbase-cluster:8020</code>.</p>
+
+<p>Since the MR Job of cube building access both main cluster and hbase cluster, exceptions may be threw during task running. The reason is task need to access HBase cluster with NN-HA related configs, which usually not deployed in the main cluster.</p>
+
+<p>These exceptions could be resolved by configs updating. In details, merge NN-HA related configs of two clusters, including <code class="highlighter-rouge">dfs.nameservices</code>, <code class="highlighter-rouge">dfs.ha.namenodes.</code>, <code class="highlighter-rouge">dfs.client.failover.proxy.provider.</code> and other related. These configs should be updated into hdfs-site.xml in Kylin Server and Resource Manager of main cluster, also be added into kylin_job_conf.xml.</p>
+
+<p>Howerver, it may be difficult to update configs in production environments. Here’s another way which avoing configs updating:</p>
<ul>
- <li>Add all <code class="highlighter-rouge">dfs.nameservices</code> related configs of HBase Cluster into <code class="highlighter-rouge">hadoop/etc/hadoop/hdfs-site.xml</code> in Kylin Server, to make sure that can access HBase Cluster using hdfs shell with nameservice path</li>
- <li>Add all <code class="highlighter-rouge">dfs.nameservices</code> related configs of both two clusters into <code class="highlighter-rouge">kylin_job_conf.xml</code>, to make sure that the MR job can access hbase cluster with nameservice path</li>
+ <li>Update HBase client in Kylin Server with patch HBASE-14347, and set <code class="highlighter-rouge">hbase.use.dynamic.jar=false</code> in hbase-site.xml</li>
+ <li>Update Kylin Server with patch KYLIN-1910, and set <code class="highlighter-rouge">kylin.hbase.cluster.hdfs.config.file=hbase.hdfs.xml</code> in kylin.properties. The <code class="highlighter-rouge">hbase.hdfs.xml</code> meaning the hdfs-site.xml of HBase cluster, put it in the same dir of kylin.properties</li>
+ <li>Update Hadoop of Kylin Server and Resource Manager with patch YARN-3021</li>
</ul>
<h2 id="troubleshooting">TroubleShooting</h2>
+
<ul>
<li>UnknownHostException occurs during Cube Building<br />
-It usually occurs with HBase HA nameservice config, please refer the above section “Using NN HA”</li>
+It usually occurs with HBase HA nameservice config, please refer the above section “Enable NN HA”</li>
+ <li>‘Error when open connection hbase’ during Kylin Server startup<br />
+HBase tries to mkdir tmp dirs on hdfs during connection setup, failed with lack of NN-HA related configs, refer “Enable NN HA”</li>
+ <li>Failed to submit cube building job<br />
+HBase cluster path will be parsed during generate cube building job, failed with lack of NN-HA related configs, refer “Enable NN HA”</li>
+ <li>Cube Building Step ‘Convert Cuboid Data to HFile’ failed<br />
+Usually occurred with Kerberos Authentication. Resource Manager tries to renew all tokens when job submitting, the renew of HBase cluster token will be failed, with lack of NN-HA related configs, refer “Enable NN HA”</li>
<li>HFile BulkLoading Stucks for long time<br />
Check the regionserver log, there should be lots of error log, with WrongFS exception. Make sure the namenode address in <code class="highlighter-rouge">kylin.properites/kylin.hbase.cluster.fs</code> and hbase master node <code class="highlighter-rouge">hbase-site.xml/root.dir</code> is same</li>
</ul>