You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ambari.apache.org by "Hadoop QA (JIRA)" <ji...@apache.org> on 2017/03/05 23:49:32 UTC
[jira] [Commented] (AMBARI-20309) HBase Master CPU Utilization
Alert is in unknown state due to kinit error
[ https://issues.apache.org/jira/browse/AMBARI-20309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896613#comment-15896613 ]
Hadoop QA commented on AMBARI-20309:
------------------------------------
{color:green}+1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12856195/AMBARI-20309_trunk_01.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in ambari-server.
Test results: https://builds.apache.org/job/Ambari-trunk-test-patch/10885//testReport/
Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/10885//console
This message is automatically generated.
> HBase Master CPU Utilization Alert is in unknown state due to kinit error
> -------------------------------------------------------------------------
>
> Key: AMBARI-20309
> URL: https://issues.apache.org/jira/browse/AMBARI-20309
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.5.0
> Reporter: Robert Levas
> Assignee: Robert Levas
> Labels: alerts, kerberos
> Fix For: 2.5.0
>
> Attachments: AMBARI-20309_branch-2.5_01.patch, AMBARI-20309_trunk_01.patch
>
>
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> {noformat}
> Execution of '/usr/bin/kinit -c /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b -kt HTTP/_HOST@EXAMPLE.COM /etc/security/keytabs/spnego.service.keytab > /dev/null' returned 1. kinit: Client not found in Kerberos database while getting initial credentials
> {noformat}
> This issue is also seen in /var/log/krb5kdc.log:
> {noformat}
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: /etc/security/keytabs/spnego.service.keytab@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM, Client not found in Kerberos database
> {noformat}
> *Cause*
> It appears that the HBASE alerts.json file ({{common-services/HBASE/0.96.0.2.0/alerts.json}}) has swapped values for the {{kerberos_keytab}} and {{kerberos_principal}} properties.
> {code}
> {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization of the HBase Master exceeds certain warning and critical thresholds. It checks the HBase Master JMX Servlet for the SystemCPULoad property. The threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
> "type": "METRIC",
> "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> },
> "reporting": {
> "ok": {
> "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
> "text": "{1} CPU, load {0:.1%}",
> "value": 200
> },
> "critical": {
> "text": "{1} CPU, load {0:.1%}",
> "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
> },
> "jmx": {
> "property_list": [
> "java.lang:type=OperatingSystem/SystemCpuLoad",
> "java.lang:type=OperatingSystem/AvailableProcessors"
> ],
> "value": "{0} * 100"
> }
> }
> }
> {code}
> Notice:
> {code}
> "kerberos_keytab": "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> {code}
> *Solution*
> Fix values for the {{kerberos_keytab}} and {{kerberos_principal}} properties in {{common-services/HBASE/0.96.0.2.0/alerts.json}}:
> {code}
> "kerberos_principal": "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_keytab": "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)