You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Robert Levas <rl...@hortonworks.com> on 2015/03/17 19:59:32 UTC
Review Request 32168: Hive alert on secured cluster
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32168/
-----------------------------------------------------------
Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Vitalyi Brodetskyi.
Bugs: AMBARI-10101
https://issues.apache.org/jira/browse/AMBARI-10101
Repository: ambari
Description
-------
When Kerberos is enabled, Hive components show alerts due to the following error:
```
WARNING 2015-03-16 06:01:08,253 base_alert.py:140 - [Alert][hive_metastore_process] Unable to execute alert. Execution of '/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa; ' returned 1. kinit: Keytab contains no suitable keys for ambari-qa@REALM while getting initial credentials
```
This occurs because the alert logic for Hive uses `cluster-env/smokeuser` rather than `cluster-env/smokeuser_principal_name` to get the principal name for the smoke test identity.
Diffs
-----
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py 804ddfe
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py 0fb8898
Diff: https://reviews.apache.org/r/32168/diff/
Testing
-------
Manually testing is cluster by setting the smoke user principal name to something other than `${cluster-env/smokeuser}@${realm}` and saw that alerts for Hive were working as designed.
Thanks,
Robert Levas
Re: Review Request 32168: Hive alert on secured cluster
Posted by Robert Levas <rl...@hortonworks.com>.
> On March 17, 2015, 3:26 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py, line 94
> > <https://reviews.apache.org/r/32168/diff/1/?file=897976#file897976line94>
> >
> > Can you explain how the smokeuser is different than the smokeuser principal? Should the commands be executed as the principal instead of the user?
>
> Robert Levas wrote:
> Esseitnally they represent two different identites. The _smoke user principal_ is the "username" of the Kerberos identity where the _smoke user_ is the "username" of the local user account. These values (related to the smoke user) tend to look a lot alike (_ambari-qa@EXAMPLE.COM_ and _ambari-qa_) but there are no rules that state that these names need to be so similar. This is where the _auto-to-local_ maps come into play. It attempts to match a principal name to a local user account, so there is no need to directly map the two.
>
> Given this, there are scenarios where the principal name of the smoke user Kerberos identity **must** be changed from _ambari-qa@REALM_. A common one is a shared KDC amoung multiple Ambari clusters. In this case, we need to make sure that one cluster does not step on another cluster's smoke user identitiy. For this, we recommend adding the cluster name to the smoke user principal. So _ambari-qa@REALM_ becomes _ambari-qa-c1@REALM_, _ambari-qa-c2@REALM_, etc...
>
> This is actually the case for all _headless_ (or _user_) identities to ensure uniqueness. For _service_ identities, the uniqueness comes in the form of a hostname, tying the identity to a particular host.
>
> Jonathan Hurley wrote:
> Thanks!
No problem... thanks for the review. :)
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32168/#review76778
-----------------------------------------------------------
On March 17, 2015, 2:59 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32168/
> -----------------------------------------------------------
>
> (Updated March 17, 2015, 2:59 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-10101
> https://issues.apache.org/jira/browse/AMBARI-10101
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When Kerberos is enabled, Hive components show alerts due to the following error:
>
> ```
> WARNING 2015-03-16 06:01:08,253 base_alert.py:140 - [Alert][hive_metastore_process] Unable to execute alert. Execution of '/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa; ' returned 1. kinit: Keytab contains no suitable keys for ambari-qa@REALM while getting initial credentials
> ```
>
> This occurs because the alert logic for Hive uses `cluster-env/smokeuser` rather than `cluster-env/smokeuser_principal_name` to get the principal name for the smoke test identity.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py 804ddfe
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py 0fb8898
>
> Diff: https://reviews.apache.org/r/32168/diff/
>
>
> Testing
> -------
>
> Manually testing is cluster by setting the smoke user principal name to something other than `${cluster-env/smokeuser}@${realm}` and saw that alerts for Hive were working as designed.
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 32168: Hive alert on secured cluster
Posted by Robert Levas <rl...@hortonworks.com>.
> On March 17, 2015, 3:26 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py, line 94
> > <https://reviews.apache.org/r/32168/diff/1/?file=897976#file897976line94>
> >
> > Can you explain how the smokeuser is different than the smokeuser principal? Should the commands be executed as the principal instead of the user?
Esseitnally they represent two different identites. The _smoke user principal_ is the "username" of the Kerberos identity where the _smoke user_ is the "username" of the local user account. These values (related to the smoke user) tend to look a lot alike (_ambari-qa@EXAMPLE.COM_ and _ambari-qa_) but there are no rules that state that these names need to be so similar. This is where the _auto-to-local_ maps come into play. It attempts to match a principal name to a local user account, so there is no need to directly map the two.
Given this, there are scenarios where the principal name of the smoke user Kerberos identity **must** be changed from _ambari-qa@REALM_. A common one is a shared KDC amoung multiple Ambari clusters. In this case, we need to make sure that one cluster does not step on another cluster's smoke user identitiy. For this, we recommend adding the cluster name to the smoke user principal. So _ambari-qa@REALM_ becomes _ambari-qa-c1@REALM_, _ambari-qa-c2@REALM_, etc...
This is actually the case for all _headless_ (or _user_) identities to ensure uniqueness. For _service_ identities, the uniqueness comes in the form of a hostname, tying the identity to a particular host.
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32168/#review76778
-----------------------------------------------------------
On March 17, 2015, 2:59 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32168/
> -----------------------------------------------------------
>
> (Updated March 17, 2015, 2:59 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-10101
> https://issues.apache.org/jira/browse/AMBARI-10101
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When Kerberos is enabled, Hive components show alerts due to the following error:
>
> ```
> WARNING 2015-03-16 06:01:08,253 base_alert.py:140 - [Alert][hive_metastore_process] Unable to execute alert. Execution of '/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa; ' returned 1. kinit: Keytab contains no suitable keys for ambari-qa@REALM while getting initial credentials
> ```
>
> This occurs because the alert logic for Hive uses `cluster-env/smokeuser` rather than `cluster-env/smokeuser_principal_name` to get the principal name for the smoke test identity.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py 804ddfe
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py 0fb8898
>
> Diff: https://reviews.apache.org/r/32168/diff/
>
>
> Testing
> -------
>
> Manually testing is cluster by setting the smoke user principal name to something other than `${cluster-env/smokeuser}@${realm}` and saw that alerts for Hive were working as designed.
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 32168: Hive alert on secured cluster
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On March 17, 2015, 3:26 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py, line 94
> > <https://reviews.apache.org/r/32168/diff/1/?file=897976#file897976line94>
> >
> > Can you explain how the smokeuser is different than the smokeuser principal? Should the commands be executed as the principal instead of the user?
>
> Robert Levas wrote:
> Esseitnally they represent two different identites. The _smoke user principal_ is the "username" of the Kerberos identity where the _smoke user_ is the "username" of the local user account. These values (related to the smoke user) tend to look a lot alike (_ambari-qa@EXAMPLE.COM_ and _ambari-qa_) but there are no rules that state that these names need to be so similar. This is where the _auto-to-local_ maps come into play. It attempts to match a principal name to a local user account, so there is no need to directly map the two.
>
> Given this, there are scenarios where the principal name of the smoke user Kerberos identity **must** be changed from _ambari-qa@REALM_. A common one is a shared KDC amoung multiple Ambari clusters. In this case, we need to make sure that one cluster does not step on another cluster's smoke user identitiy. For this, we recommend adding the cluster name to the smoke user principal. So _ambari-qa@REALM_ becomes _ambari-qa-c1@REALM_, _ambari-qa-c2@REALM_, etc...
>
> This is actually the case for all _headless_ (or _user_) identities to ensure uniqueness. For _service_ identities, the uniqueness comes in the form of a hostname, tying the identity to a particular host.
Thanks!
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32168/#review76778
-----------------------------------------------------------
On March 17, 2015, 2:59 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32168/
> -----------------------------------------------------------
>
> (Updated March 17, 2015, 2:59 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-10101
> https://issues.apache.org/jira/browse/AMBARI-10101
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When Kerberos is enabled, Hive components show alerts due to the following error:
>
> ```
> WARNING 2015-03-16 06:01:08,253 base_alert.py:140 - [Alert][hive_metastore_process] Unable to execute alert. Execution of '/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa; ' returned 1. kinit: Keytab contains no suitable keys for ambari-qa@REALM while getting initial credentials
> ```
>
> This occurs because the alert logic for Hive uses `cluster-env/smokeuser` rather than `cluster-env/smokeuser_principal_name` to get the principal name for the smoke test identity.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py 804ddfe
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py 0fb8898
>
> Diff: https://reviews.apache.org/r/32168/diff/
>
>
> Testing
> -------
>
> Manually testing is cluster by setting the smoke user principal name to something other than `${cluster-env/smokeuser}@${realm}` and saw that alerts for Hive were working as designed.
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 32168: Hive alert on secured cluster
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32168/#review76778
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py
<https://reviews.apache.org/r/32168/#comment124446>
Can you explain how the smokeuser is different than the smokeuser principal? Should the commands be executed as the principal instead of the user?
- Jonathan Hurley
On March 17, 2015, 2:59 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32168/
> -----------------------------------------------------------
>
> (Updated March 17, 2015, 2:59 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-10101
> https://issues.apache.org/jira/browse/AMBARI-10101
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When Kerberos is enabled, Hive components show alerts due to the following error:
>
> ```
> WARNING 2015-03-16 06:01:08,253 base_alert.py:140 - [Alert][hive_metastore_process] Unable to execute alert. Execution of '/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa; ' returned 1. kinit: Keytab contains no suitable keys for ambari-qa@REALM while getting initial credentials
> ```
>
> This occurs because the alert logic for Hive uses `cluster-env/smokeuser` rather than `cluster-env/smokeuser_principal_name` to get the principal name for the smoke test identity.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py 804ddfe
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py 0fb8898
>
> Diff: https://reviews.apache.org/r/32168/diff/
>
>
> Testing
> -------
>
> Manually testing is cluster by setting the smoke user principal name to something other than `${cluster-env/smokeuser}@${realm}` and saw that alerts for Hive were working as designed.
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 32168: Hive alert on secured cluster
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32168/#review76792
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On March 17, 2015, 2:59 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32168/
> -----------------------------------------------------------
>
> (Updated March 17, 2015, 2:59 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-10101
> https://issues.apache.org/jira/browse/AMBARI-10101
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When Kerberos is enabled, Hive components show alerts due to the following error:
>
> ```
> WARNING 2015-03-16 06:01:08,253 base_alert.py:140 - [Alert][hive_metastore_process] Unable to execute alert. Execution of '/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa; ' returned 1. kinit: Keytab contains no suitable keys for ambari-qa@REALM while getting initial credentials
> ```
>
> This occurs because the alert logic for Hive uses `cluster-env/smokeuser` rather than `cluster-env/smokeuser_principal_name` to get the principal name for the smoke test identity.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py 804ddfe
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py 0fb8898
>
> Diff: https://reviews.apache.org/r/32168/diff/
>
>
> Testing
> -------
>
> Manually testing is cluster by setting the smoke user principal name to something other than `${cluster-env/smokeuser}@${realm}` and saw that alerts for Hive were working as designed.
>
>
> Thanks,
>
> Robert Levas
>
>