You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Qin Liu <qi...@gmail.com> on 2017/08/30 19:07:18 UTC
Review Request 61998: Update Ambari Metrics service check to support
HTTP SPNEGO authentication
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61998/
-----------------------------------------------------------
Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Eric Yang, Robert Levas, and Sid Wagle.
Bugs: AMBARI-21821
https://issues.apache.org/jira/browse/AMBARI-21821
Repository: ambari
Description
-------
This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics service check currently does not support Kerberos HTTP SPNEGO authentication.
2017-08-24 16:13:44,078 - Ambari Metrics service check was started.
2017-08-24 16:13:44,109 - Generated metrics for host amshost.example.com :
{
"metrics": [
{
"metricname": "AMBARI_METRICS.SmokeTest.FakeMetric",
"appid": "amssmoketestfake",
"hostname": "host.example.com",
"timestamp": 1503616424000,
"starttime": 1503616424000,
"metrics":
{ "1503616424000": 0.164134766408, "1503616425000": 1503616424000 }
}
]
}
2017-08-24 16:13:44,114 - Connecting (POST) to amshost.example.com:6188/ws/v1/timeline/metrics/
2017-08-24 16:13:44,117 - Http response for host amshost.example.com: 401 Authentication required
Diffs
-----
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 6975bec
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py 2b3dfa9
Diff: https://reviews.apache.org/r/61998/diff/1/
Testing
-------
1. run the Hadoop QA testing twice and the testcase failures has nothing to do with the patch
2. manually tested AMS service check on HDP/trunk cluster with combination of HTTP SPNEGO/Non-HTTP SPNEGO, SSL/Non-SSL, AMS distributed/embedded.
Thanks,
Qin Liu
Re: Review Request 61998: Update Ambari Metrics service check to
support HTTP SPNEGO authentication
Posted by Qin Liu <qi...@gmail.com>.
> On Sept. 1, 2017, 12:41 p.m., Robert Levas wrote:
> > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
> > Lines 182-190 (patched)
> > <https://reviews.apache.org/r/61998/diff/1/?file=1807987#file1807987line185>
> >
> > This can be simplified to
> >
> > ```
> > return params.security_enabled \
> > and 'core-site' in params.config['configurations'] \
> > and 'hadoop.http.authentication.type' in params.config['configurations']['core-site'] \
> > and params.config['configurations']['core-site']['hadoop.http.authentication.type'] == "kerberos" \
> > and 'hadoop.http.filter.initializers' in params.config['configurations']['core-site'] \
> > and params.config['configurations']['core-site']['hadoop.http.filter.initializers'] == "org.apache.hadoop.security.AuthenticationFilterInitializer"
> > ```
Hi Robert, Thank you so much for the review! I will fix it!
- Qin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61998/#review184351
-----------------------------------------------------------
On Aug. 30, 2017, 7:07 p.m., Qin Liu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61998/
> -----------------------------------------------------------
>
> (Updated Aug. 30, 2017, 7:07 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Eric Yang, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-21821
> https://issues.apache.org/jira/browse/AMBARI-21821
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics service check currently does not support Kerberos HTTP SPNEGO authentication.
>
> 2017-08-24 16:13:44,078 - Ambari Metrics service check was started.
> 2017-08-24 16:13:44,109 - Generated metrics for host amshost.example.com :
> {
> "metrics": [
> {
> "metricname": "AMBARI_METRICS.SmokeTest.FakeMetric",
> "appid": "amssmoketestfake",
> "hostname": "host.example.com",
> "timestamp": 1503616424000,
> "starttime": 1503616424000,
> "metrics":
> { "1503616424000": 0.164134766408, "1503616425000": 1503616424000 }
>
> }
> ]
> }
> 2017-08-24 16:13:44,114 - Connecting (POST) to amshost.example.com:6188/ws/v1/timeline/metrics/
> 2017-08-24 16:13:44,117 - Http response for host amshost.example.com: 401 Authentication required
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 6975bec
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py 2b3dfa9
>
>
> Diff: https://reviews.apache.org/r/61998/diff/1/
>
>
> Testing
> -------
>
> 1. run the Hadoop QA testing twice and the testcase failures has nothing to do with the patch
>
> 2. manually tested AMS service check on HDP/trunk cluster with combination of HTTP SPNEGO/Non-HTTP SPNEGO, SSL/Non-SSL, AMS distributed/embedded.
>
>
> Thanks,
>
> Qin Liu
>
>
Re: Review Request 61998: Update Ambari Metrics service check to
support HTTP SPNEGO authentication
Posted by Qin Liu <qi...@gmail.com>.
> On Sept. 1, 2017, 12:41 p.m., Robert Levas wrote:
> > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
> > Lines 182-190 (patched)
> > <https://reviews.apache.org/r/61998/diff/1/?file=1807987#file1807987line185>
> >
> > This can be simplified to
> >
> > ```
> > return params.security_enabled \
> > and 'core-site' in params.config['configurations'] \
> > and 'hadoop.http.authentication.type' in params.config['configurations']['core-site'] \
> > and params.config['configurations']['core-site']['hadoop.http.authentication.type'] == "kerberos" \
> > and 'hadoop.http.filter.initializers' in params.config['configurations']['core-site'] \
> > and params.config['configurations']['core-site']['hadoop.http.filter.initializers'] == "org.apache.hadoop.security.AuthenticationFilterInitializer"
> > ```
>
> Qin Liu wrote:
> Hi Robert, Thank you so much for the review! I will fix it!
Hi Robert, The new patch has been updated with your suggestion! Thank you very much!
- Qin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61998/#review184351
-----------------------------------------------------------
On Sept. 5, 2017, 7:50 p.m., Qin Liu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61998/
> -----------------------------------------------------------
>
> (Updated Sept. 5, 2017, 7:50 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Eric Yang, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-21821
> https://issues.apache.org/jira/browse/AMBARI-21821
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics service check currently does not support Kerberos HTTP SPNEGO authentication.
>
> 2017-08-24 16:13:44,078 - Ambari Metrics service check was started.
> 2017-08-24 16:13:44,109 - Generated metrics for host amshost.example.com :
> {
> "metrics": [
> {
> "metricname": "AMBARI_METRICS.SmokeTest.FakeMetric",
> "appid": "amssmoketestfake",
> "hostname": "host.example.com",
> "timestamp": 1503616424000,
> "starttime": 1503616424000,
> "metrics":
> { "1503616424000": 0.164134766408, "1503616425000": 1503616424000 }
>
> }
> ]
> }
> 2017-08-24 16:13:44,114 - Connecting (POST) to amshost.example.com:6188/ws/v1/timeline/metrics/
> 2017-08-24 16:13:44,117 - Http response for host amshost.example.com: 401 Authentication required
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 6975bec
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py 2b3dfa9
>
>
> Diff: https://reviews.apache.org/r/61998/diff/2/
>
>
> Testing
> -------
>
> 1. run the Hadoop QA testing twice and the testcase failures has nothing to do with the patch
>
> 2. manually tested AMS service check on HDP/trunk cluster with combination of HTTP SPNEGO/Non-HTTP SPNEGO, SSL/Non-SSL, AMS distributed/embedded.
>
>
> File Attachments
> ----------------
>
> AMBARI-21821_2.patch
> https://reviews.apache.org/media/uploaded/files/2017/09/05/7241c0b4-7c48-47b1-b45c-c1ec1d71d4e0__AMBARI-21821_2.patch
>
>
> Thanks,
>
> Qin Liu
>
>
Re: Review Request 61998: Update Ambari Metrics service check to
support HTTP SPNEGO authentication
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61998/#review184351
-----------------------------------------------------------
Ship it!
Ship It!
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
Lines 182-190 (patched)
<https://reviews.apache.org/r/61998/#comment260484>
This can be simplified to
```
return params.security_enabled \
and 'core-site' in params.config['configurations'] \
and 'hadoop.http.authentication.type' in params.config['configurations']['core-site'] \
and params.config['configurations']['core-site']['hadoop.http.authentication.type'] == "kerberos" \
and 'hadoop.http.filter.initializers' in params.config['configurations']['core-site'] \
and params.config['configurations']['core-site']['hadoop.http.filter.initializers'] == "org.apache.hadoop.security.AuthenticationFilterInitializer"
```
- Robert Levas
On Aug. 30, 2017, 3:07 p.m., Qin Liu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61998/
> -----------------------------------------------------------
>
> (Updated Aug. 30, 2017, 3:07 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Eric Yang, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-21821
> https://issues.apache.org/jira/browse/AMBARI-21821
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics service check currently does not support Kerberos HTTP SPNEGO authentication.
>
> 2017-08-24 16:13:44,078 - Ambari Metrics service check was started.
> 2017-08-24 16:13:44,109 - Generated metrics for host amshost.example.com :
> {
> "metrics": [
> {
> "metricname": "AMBARI_METRICS.SmokeTest.FakeMetric",
> "appid": "amssmoketestfake",
> "hostname": "host.example.com",
> "timestamp": 1503616424000,
> "starttime": 1503616424000,
> "metrics":
> { "1503616424000": 0.164134766408, "1503616425000": 1503616424000 }
>
> }
> ]
> }
> 2017-08-24 16:13:44,114 - Connecting (POST) to amshost.example.com:6188/ws/v1/timeline/metrics/
> 2017-08-24 16:13:44,117 - Http response for host amshost.example.com: 401 Authentication required
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 6975bec
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py 2b3dfa9
>
>
> Diff: https://reviews.apache.org/r/61998/diff/1/
>
>
> Testing
> -------
>
> 1. run the Hadoop QA testing twice and the testcase failures has nothing to do with the patch
>
> 2. manually tested AMS service check on HDP/trunk cluster with combination of HTTP SPNEGO/Non-HTTP SPNEGO, SSL/Non-SSL, AMS distributed/embedded.
>
>
> Thanks,
>
> Qin Liu
>
>
Re: Review Request 61998: Update Ambari Metrics service check to
support HTTP SPNEGO authentication
Posted by Qin Liu <qi...@gmail.com>.
> On Sept. 7, 2017, 10:21 p.m., Eric Yang wrote:
> > Ship It!
Hi Eric, Thank you so much!
- Qin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61998/#review184894
-----------------------------------------------------------
On Sept. 5, 2017, 7:50 p.m., Qin Liu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61998/
> -----------------------------------------------------------
>
> (Updated Sept. 5, 2017, 7:50 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Eric Yang, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-21821
> https://issues.apache.org/jira/browse/AMBARI-21821
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics service check currently does not support Kerberos HTTP SPNEGO authentication.
>
> 2017-08-24 16:13:44,078 - Ambari Metrics service check was started.
> 2017-08-24 16:13:44,109 - Generated metrics for host amshost.example.com :
> {
> "metrics": [
> {
> "metricname": "AMBARI_METRICS.SmokeTest.FakeMetric",
> "appid": "amssmoketestfake",
> "hostname": "host.example.com",
> "timestamp": 1503616424000,
> "starttime": 1503616424000,
> "metrics":
> { "1503616424000": 0.164134766408, "1503616425000": 1503616424000 }
>
> }
> ]
> }
> 2017-08-24 16:13:44,114 - Connecting (POST) to amshost.example.com:6188/ws/v1/timeline/metrics/
> 2017-08-24 16:13:44,117 - Http response for host amshost.example.com: 401 Authentication required
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 6975bec
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py 2b3dfa9
>
>
> Diff: https://reviews.apache.org/r/61998/diff/2/
>
>
> Testing
> -------
>
> 1. run the Hadoop QA testing twice and the testcase failures has nothing to do with the patch
>
> 2. manually tested AMS service check on HDP/trunk cluster with combination of HTTP SPNEGO/Non-HTTP SPNEGO, SSL/Non-SSL, AMS distributed/embedded.
>
>
> File Attachments
> ----------------
>
> AMBARI-21821_2.patch
> https://reviews.apache.org/media/uploaded/files/2017/09/05/7241c0b4-7c48-47b1-b45c-c1ec1d71d4e0__AMBARI-21821_2.patch
>
>
> Thanks,
>
> Qin Liu
>
>
Re: Review Request 61998: Update Ambari Metrics service check to
support HTTP SPNEGO authentication
Posted by Eric Yang <ey...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61998/#review184894
-----------------------------------------------------------
Ship it!
Ship It!
- Eric Yang
On Sept. 5, 2017, 7:50 p.m., Qin Liu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61998/
> -----------------------------------------------------------
>
> (Updated Sept. 5, 2017, 7:50 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Eric Yang, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-21821
> https://issues.apache.org/jira/browse/AMBARI-21821
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics service check currently does not support Kerberos HTTP SPNEGO authentication.
>
> 2017-08-24 16:13:44,078 - Ambari Metrics service check was started.
> 2017-08-24 16:13:44,109 - Generated metrics for host amshost.example.com :
> {
> "metrics": [
> {
> "metricname": "AMBARI_METRICS.SmokeTest.FakeMetric",
> "appid": "amssmoketestfake",
> "hostname": "host.example.com",
> "timestamp": 1503616424000,
> "starttime": 1503616424000,
> "metrics":
> { "1503616424000": 0.164134766408, "1503616425000": 1503616424000 }
>
> }
> ]
> }
> 2017-08-24 16:13:44,114 - Connecting (POST) to amshost.example.com:6188/ws/v1/timeline/metrics/
> 2017-08-24 16:13:44,117 - Http response for host amshost.example.com: 401 Authentication required
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 6975bec
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py 2b3dfa9
>
>
> Diff: https://reviews.apache.org/r/61998/diff/2/
>
>
> Testing
> -------
>
> 1. run the Hadoop QA testing twice and the testcase failures has nothing to do with the patch
>
> 2. manually tested AMS service check on HDP/trunk cluster with combination of HTTP SPNEGO/Non-HTTP SPNEGO, SSL/Non-SSL, AMS distributed/embedded.
>
>
> File Attachments
> ----------------
>
> AMBARI-21821_2.patch
> https://reviews.apache.org/media/uploaded/files/2017/09/05/7241c0b4-7c48-47b1-b45c-c1ec1d71d4e0__AMBARI-21821_2.patch
>
>
> Thanks,
>
> Qin Liu
>
>
Re: Review Request 61998: Update Ambari Metrics service check to
support HTTP SPNEGO authentication
Posted by Qin Liu <qi...@gmail.com>.
> On Sept. 8, 2017, 3:46 a.m., Aravindan Vijayan wrote:
> > Ship It!
Hi Aravindan, Thank you so much!
- Qin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61998/#review184933
-----------------------------------------------------------
On Sept. 5, 2017, 7:50 p.m., Qin Liu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61998/
> -----------------------------------------------------------
>
> (Updated Sept. 5, 2017, 7:50 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Eric Yang, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-21821
> https://issues.apache.org/jira/browse/AMBARI-21821
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics service check currently does not support Kerberos HTTP SPNEGO authentication.
>
> 2017-08-24 16:13:44,078 - Ambari Metrics service check was started.
> 2017-08-24 16:13:44,109 - Generated metrics for host amshost.example.com :
> {
> "metrics": [
> {
> "metricname": "AMBARI_METRICS.SmokeTest.FakeMetric",
> "appid": "amssmoketestfake",
> "hostname": "host.example.com",
> "timestamp": 1503616424000,
> "starttime": 1503616424000,
> "metrics":
> { "1503616424000": 0.164134766408, "1503616425000": 1503616424000 }
>
> }
> ]
> }
> 2017-08-24 16:13:44,114 - Connecting (POST) to amshost.example.com:6188/ws/v1/timeline/metrics/
> 2017-08-24 16:13:44,117 - Http response for host amshost.example.com: 401 Authentication required
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 6975bec
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py 2b3dfa9
>
>
> Diff: https://reviews.apache.org/r/61998/diff/2/
>
>
> Testing
> -------
>
> 1. run the Hadoop QA testing twice and the testcase failures has nothing to do with the patch
>
> 2. manually tested AMS service check on HDP/trunk cluster with combination of HTTP SPNEGO/Non-HTTP SPNEGO, SSL/Non-SSL, AMS distributed/embedded.
>
>
> File Attachments
> ----------------
>
> AMBARI-21821_2.patch
> https://reviews.apache.org/media/uploaded/files/2017/09/05/7241c0b4-7c48-47b1-b45c-c1ec1d71d4e0__AMBARI-21821_2.patch
>
>
> Thanks,
>
> Qin Liu
>
>
Re: Review Request 61998: Update Ambari Metrics service check to
support HTTP SPNEGO authentication
Posted by Aravindan Vijayan <av...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61998/#review184933
-----------------------------------------------------------
Ship it!
Ship It!
- Aravindan Vijayan
On Sept. 5, 2017, 7:50 p.m., Qin Liu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61998/
> -----------------------------------------------------------
>
> (Updated Sept. 5, 2017, 7:50 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Eric Yang, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-21821
> https://issues.apache.org/jira/browse/AMBARI-21821
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics service check currently does not support Kerberos HTTP SPNEGO authentication.
>
> 2017-08-24 16:13:44,078 - Ambari Metrics service check was started.
> 2017-08-24 16:13:44,109 - Generated metrics for host amshost.example.com :
> {
> "metrics": [
> {
> "metricname": "AMBARI_METRICS.SmokeTest.FakeMetric",
> "appid": "amssmoketestfake",
> "hostname": "host.example.com",
> "timestamp": 1503616424000,
> "starttime": 1503616424000,
> "metrics":
> { "1503616424000": 0.164134766408, "1503616425000": 1503616424000 }
>
> }
> ]
> }
> 2017-08-24 16:13:44,114 - Connecting (POST) to amshost.example.com:6188/ws/v1/timeline/metrics/
> 2017-08-24 16:13:44,117 - Http response for host amshost.example.com: 401 Authentication required
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 6975bec
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py 2b3dfa9
>
>
> Diff: https://reviews.apache.org/r/61998/diff/2/
>
>
> Testing
> -------
>
> 1. run the Hadoop QA testing twice and the testcase failures has nothing to do with the patch
>
> 2. manually tested AMS service check on HDP/trunk cluster with combination of HTTP SPNEGO/Non-HTTP SPNEGO, SSL/Non-SSL, AMS distributed/embedded.
>
>
> File Attachments
> ----------------
>
> AMBARI-21821_2.patch
> https://reviews.apache.org/media/uploaded/files/2017/09/05/7241c0b4-7c48-47b1-b45c-c1ec1d71d4e0__AMBARI-21821_2.patch
>
>
> Thanks,
>
> Qin Liu
>
>
Re: Review Request 61998: Update Ambari Metrics service check to
support HTTP SPNEGO authentication
Posted by Qin Liu <qi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61998/
-----------------------------------------------------------
(Updated Sept. 5, 2017, 7:50 p.m.)
Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Eric Yang, Robert Levas, and Sid Wagle.
Bugs: AMBARI-21821
https://issues.apache.org/jira/browse/AMBARI-21821
Repository: ambari
Description
-------
This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics service check currently does not support Kerberos HTTP SPNEGO authentication.
2017-08-24 16:13:44,078 - Ambari Metrics service check was started.
2017-08-24 16:13:44,109 - Generated metrics for host amshost.example.com :
{
"metrics": [
{
"metricname": "AMBARI_METRICS.SmokeTest.FakeMetric",
"appid": "amssmoketestfake",
"hostname": "host.example.com",
"timestamp": 1503616424000,
"starttime": 1503616424000,
"metrics":
{ "1503616424000": 0.164134766408, "1503616425000": 1503616424000 }
}
]
}
2017-08-24 16:13:44,114 - Connecting (POST) to amshost.example.com:6188/ws/v1/timeline/metrics/
2017-08-24 16:13:44,117 - Http response for host amshost.example.com: 401 Authentication required
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 6975bec
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py 2b3dfa9
Diff: https://reviews.apache.org/r/61998/diff/2/
Changes: https://reviews.apache.org/r/61998/diff/1-2/
Testing
-------
1. run the Hadoop QA testing twice and the testcase failures has nothing to do with the patch
2. manually tested AMS service check on HDP/trunk cluster with combination of HTTP SPNEGO/Non-HTTP SPNEGO, SSL/Non-SSL, AMS distributed/embedded.
File Attachments (updated)
----------------
AMBARI-21821_2.patch
https://reviews.apache.org/media/uploaded/files/2017/09/05/7241c0b4-7c48-47b1-b45c-c1ec1d71d4e0__AMBARI-21821_2.patch
Thanks,
Qin Liu
Re: Review Request 61998: Update Ambari Metrics service check to
support HTTP SPNEGO authentication
Posted by Qin Liu <qi...@gmail.com>.
> On Aug. 31, 2017, 10:23 p.m., Eric Yang wrote:
> > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
> > Lines 80 (patched)
> > <https://reviews.apache.org/r/61998/diff/1/?file=1807987#file1807987line83>
> >
> > I would recommend to avoid conditional formatting using input that came from user. This could trigger security hole, see: http://lucumr.pocoo.org/2016/12/29/careful-with-str-format/
>
> Qin Liu wrote:
> Hi Eric, Thank you so much for your suggestion! I will investigate it.
Hi Eric, The new patch has been updated to reflect the suggested fix. Thank you very much!
- Qin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61998/#review184319
-----------------------------------------------------------
On Sept. 5, 2017, 7:50 p.m., Qin Liu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61998/
> -----------------------------------------------------------
>
> (Updated Sept. 5, 2017, 7:50 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Eric Yang, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-21821
> https://issues.apache.org/jira/browse/AMBARI-21821
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics service check currently does not support Kerberos HTTP SPNEGO authentication.
>
> 2017-08-24 16:13:44,078 - Ambari Metrics service check was started.
> 2017-08-24 16:13:44,109 - Generated metrics for host amshost.example.com :
> {
> "metrics": [
> {
> "metricname": "AMBARI_METRICS.SmokeTest.FakeMetric",
> "appid": "amssmoketestfake",
> "hostname": "host.example.com",
> "timestamp": 1503616424000,
> "starttime": 1503616424000,
> "metrics":
> { "1503616424000": 0.164134766408, "1503616425000": 1503616424000 }
>
> }
> ]
> }
> 2017-08-24 16:13:44,114 - Connecting (POST) to amshost.example.com:6188/ws/v1/timeline/metrics/
> 2017-08-24 16:13:44,117 - Http response for host amshost.example.com: 401 Authentication required
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 6975bec
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py 2b3dfa9
>
>
> Diff: https://reviews.apache.org/r/61998/diff/2/
>
>
> Testing
> -------
>
> 1. run the Hadoop QA testing twice and the testcase failures has nothing to do with the patch
>
> 2. manually tested AMS service check on HDP/trunk cluster with combination of HTTP SPNEGO/Non-HTTP SPNEGO, SSL/Non-SSL, AMS distributed/embedded.
>
>
> File Attachments
> ----------------
>
> AMBARI-21821_2.patch
> https://reviews.apache.org/media/uploaded/files/2017/09/05/7241c0b4-7c48-47b1-b45c-c1ec1d71d4e0__AMBARI-21821_2.patch
>
>
> Thanks,
>
> Qin Liu
>
>
Re: Review Request 61998: Update Ambari Metrics service check to
support HTTP SPNEGO authentication
Posted by Qin Liu <qi...@gmail.com>.
> On Aug. 31, 2017, 10:23 p.m., Eric Yang wrote:
> > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
> > Lines 80 (patched)
> > <https://reviews.apache.org/r/61998/diff/1/?file=1807987#file1807987line83>
> >
> > I would recommend to avoid conditional formatting using input that came from user. This could trigger security hole, see: http://lucumr.pocoo.org/2016/12/29/careful-with-str-format/
Hi Eric, Thank you so much for your suggestion! I will investigate it.
- Qin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61998/#review184319
-----------------------------------------------------------
On Aug. 30, 2017, 7:07 p.m., Qin Liu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61998/
> -----------------------------------------------------------
>
> (Updated Aug. 30, 2017, 7:07 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Eric Yang, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-21821
> https://issues.apache.org/jira/browse/AMBARI-21821
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics service check currently does not support Kerberos HTTP SPNEGO authentication.
>
> 2017-08-24 16:13:44,078 - Ambari Metrics service check was started.
> 2017-08-24 16:13:44,109 - Generated metrics for host amshost.example.com :
> {
> "metrics": [
> {
> "metricname": "AMBARI_METRICS.SmokeTest.FakeMetric",
> "appid": "amssmoketestfake",
> "hostname": "host.example.com",
> "timestamp": 1503616424000,
> "starttime": 1503616424000,
> "metrics":
> { "1503616424000": 0.164134766408, "1503616425000": 1503616424000 }
>
> }
> ]
> }
> 2017-08-24 16:13:44,114 - Connecting (POST) to amshost.example.com:6188/ws/v1/timeline/metrics/
> 2017-08-24 16:13:44,117 - Http response for host amshost.example.com: 401 Authentication required
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 6975bec
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py 2b3dfa9
>
>
> Diff: https://reviews.apache.org/r/61998/diff/1/
>
>
> Testing
> -------
>
> 1. run the Hadoop QA testing twice and the testcase failures has nothing to do with the patch
>
> 2. manually tested AMS service check on HDP/trunk cluster with combination of HTTP SPNEGO/Non-HTTP SPNEGO, SSL/Non-SSL, AMS distributed/embedded.
>
>
> Thanks,
>
> Qin Liu
>
>
Re: Review Request 61998: Update Ambari Metrics service check to
support HTTP SPNEGO authentication
Posted by Eric Yang <ey...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61998/#review184319
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
Lines 80 (patched)
<https://reviews.apache.org/r/61998/#comment260449>
I would recommend to avoid conditional formatting using input that came from user. This could trigger security hole, see: http://lucumr.pocoo.org/2016/12/29/careful-with-str-format/
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
Lines 105 (patched)
<https://reviews.apache.org/r/61998/#comment260450>
I would recommend to avoid conditional formatting using input that came from user. This could trigger security hole, see: http://lucumr.pocoo.org/2016/12/29/careful-with-str-format/
- Eric Yang
On Aug. 30, 2017, 7:07 p.m., Qin Liu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61998/
> -----------------------------------------------------------
>
> (Updated Aug. 30, 2017, 7:07 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Eric Yang, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-21821
> https://issues.apache.org/jira/browse/AMBARI-21821
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics service check currently does not support Kerberos HTTP SPNEGO authentication.
>
> 2017-08-24 16:13:44,078 - Ambari Metrics service check was started.
> 2017-08-24 16:13:44,109 - Generated metrics for host amshost.example.com :
> {
> "metrics": [
> {
> "metricname": "AMBARI_METRICS.SmokeTest.FakeMetric",
> "appid": "amssmoketestfake",
> "hostname": "host.example.com",
> "timestamp": 1503616424000,
> "starttime": 1503616424000,
> "metrics":
> { "1503616424000": 0.164134766408, "1503616425000": 1503616424000 }
>
> }
> ]
> }
> 2017-08-24 16:13:44,114 - Connecting (POST) to amshost.example.com:6188/ws/v1/timeline/metrics/
> 2017-08-24 16:13:44,117 - Http response for host amshost.example.com: 401 Authentication required
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 6975bec
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py 2b3dfa9
>
>
> Diff: https://reviews.apache.org/r/61998/diff/1/
>
>
> Testing
> -------
>
> 1. run the Hadoop QA testing twice and the testcase failures has nothing to do with the patch
>
> 2. manually tested AMS service check on HDP/trunk cluster with combination of HTTP SPNEGO/Non-HTTP SPNEGO, SSL/Non-SSL, AMS distributed/embedded.
>
>
> Thanks,
>
> Qin Liu
>
>