You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Robert Levas <rl...@hortonworks.com> on 2017/07/05 18:38:50 UTC
Re: Review Request 60403: Add Kerberos HTTP SPNEGO authentication
support to Ambari Metrics Monitor
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60403/#review179673
-----------------------------------------------------------
Fix it, then Ship it!
Ship It!
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py
Lines 33 (patched)
<https://reviews.apache.org/r/60403/#comment254487>
Would it be possible to use `resource_management.libraries.functions.curl_krb_request.curl_krb_request` instead?
- Robert Levas
On June 23, 2017, 6:49 p.m., Qin Liu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60403/
> -----------------------------------------------------------
>
> (Updated June 23, 2017, 6:49 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan, Di Li, Dmytro Sen, Robert Levas, Sid Wagle, and Tim Thorpe.
>
>
> Bugs: AMBARI-17898
> https://issues.apache.org/jira/browse/AMBARI-17898
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics Monitor, a client of Ambari Metrics Collector, currently does not support Kerberos HTTP SPNEGO authentication.
> /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out:
> 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: http://metrics-collector:6188/ws/v1/timeline/metrics
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to server. HTTP Error 401: Authentication required
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ...
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini 7fe7397
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py d1429ed
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py 77b8c23
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/krberr.py PRE-CREATION
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py PRE-CREATION
> ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestEmitter.py 4056ae3
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json 51f541f
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/metainfo.xml 9031b46
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py f49d47d
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py b8c14f4
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2 b7dee50
>
>
> Diff: https://reviews.apache.org/r/60403/diff/1/
>
>
> Testing
> -------
>
> 1. Unit tests passed
> 2. manually tested with HDP/trunk
>
>
> Thanks,
>
> Qin Liu
>
>
Re: Review Request 60403: Add Kerberos HTTP SPNEGO authentication
support to Ambari Metrics Monitor
Posted by Qin Liu <qi...@gmail.com>.
> On July 5, 2017, 6:38 p.m., Robert Levas wrote:
> > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py
> > Lines 33 (patched)
> > <https://reviews.apache.org/r/60403/diff/1/?file=1761672#file1761672line33>
> >
> > Would it be possible to use `resource_management.libraries.functions.curl_krb_request.curl_krb_request` instead?
Hi Robert,
Thank you so much for your review! I just came back from vacation and will re-visit it ASAP.
- Qin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60403/#review179673
-----------------------------------------------------------
On June 23, 2017, 10:49 p.m., Qin Liu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60403/
> -----------------------------------------------------------
>
> (Updated June 23, 2017, 10:49 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan, Di Li, Dmytro Sen, Robert Levas, Sid Wagle, and Tim Thorpe.
>
>
> Bugs: AMBARI-17898
> https://issues.apache.org/jira/browse/AMBARI-17898
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics Monitor, a client of Ambari Metrics Collector, currently does not support Kerberos HTTP SPNEGO authentication.
> /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out:
> 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: http://metrics-collector:6188/ws/v1/timeline/metrics
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to server. HTTP Error 401: Authentication required
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ...
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini 7fe7397
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py d1429ed
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py 77b8c23
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/krberr.py PRE-CREATION
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py PRE-CREATION
> ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestEmitter.py 4056ae3
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json 51f541f
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/metainfo.xml 9031b46
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py f49d47d
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py b8c14f4
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2 b7dee50
>
>
> Diff: https://reviews.apache.org/r/60403/diff/1/
>
>
> Testing
> -------
>
> 1. Unit tests passed
> 2. manually tested with HDP/trunk
>
>
> Thanks,
>
> Qin Liu
>
>
Re: Review Request 60403: Add Kerberos HTTP SPNEGO authentication
support to Ambari Metrics Monitor
Posted by Robert Levas <rl...@hortonworks.com>.
> On July 5, 2017, 2:38 p.m., Robert Levas wrote:
> > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py
> > Lines 33 (patched)
> > <https://reviews.apache.org/r/60403/diff/1/?file=1761672#file1761672line33>
> >
> > Would it be possible to use `resource_management.libraries.functions.curl_krb_request.curl_krb_request` instead?
>
> Qin Liu wrote:
> Hi Robert,
> Thank you so much for your review! I just came back from vacation and will re-visit it ASAP.
>
> Qin Liu wrote:
> Hi Robert,
>
> Sorry for the delay!
>
> I think spnego_kerberos_auth is better choice than curl_krb_request for metrics monitor because it will have better performance:
> 1. Metrics monitor curently uses the same cached HTTP connection for every POST call while curl_krb_request will make different HTTP connection for each call
> 2. Metrics monitor can take advantage of the same cached HTTP connection for every POST call by using spnego_kerberos_auth
> 3. With spnego_kerberos_auth, metrics monitor will use cookie in memory while curl_krb_request uses cookie cache in disk
>
> curl_krb_request can be better choice for alert monitors due to the nature of alerts - should be on different connection for each call.
Hey Qin... Thanks for the clarification. I will drop this issue.
Sorry for the delay, I was out on vacation last week and am just getting caugh up.
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60403/#review179673
-----------------------------------------------------------
On June 23, 2017, 6:49 p.m., Qin Liu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60403/
> -----------------------------------------------------------
>
> (Updated June 23, 2017, 6:49 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan, Di Li, Dmytro Sen, Robert Levas, Sid Wagle, and Tim Thorpe.
>
>
> Bugs: AMBARI-17898
> https://issues.apache.org/jira/browse/AMBARI-17898
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics Monitor, a client of Ambari Metrics Collector, currently does not support Kerberos HTTP SPNEGO authentication.
> /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out:
> 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: http://metrics-collector:6188/ws/v1/timeline/metrics
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to server. HTTP Error 401: Authentication required
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ...
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini 7fe7397
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py d1429ed
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py 77b8c23
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/krberr.py PRE-CREATION
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py PRE-CREATION
> ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestEmitter.py 4056ae3
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json 51f541f
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/metainfo.xml 9031b46
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py f49d47d
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py b8c14f4
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2 b7dee50
>
>
> Diff: https://reviews.apache.org/r/60403/diff/1/
>
>
> Testing
> -------
>
> 1. Unit tests passed
> 2. manually tested with HDP/trunk
>
>
> Thanks,
>
> Qin Liu
>
>
Re: Review Request 60403: Add Kerberos HTTP SPNEGO authentication
support to Ambari Metrics Monitor
Posted by Qin Liu <qi...@gmail.com>.
> On July 5, 2017, 6:38 p.m., Robert Levas wrote:
> > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py
> > Lines 33 (patched)
> > <https://reviews.apache.org/r/60403/diff/1/?file=1761672#file1761672line33>
> >
> > Would it be possible to use `resource_management.libraries.functions.curl_krb_request.curl_krb_request` instead?
>
> Qin Liu wrote:
> Hi Robert,
> Thank you so much for your review! I just came back from vacation and will re-visit it ASAP.
Hi Robert,
Sorry for the delay!
I think spnego_kerberos_auth is better choice than curl_krb_request for metrics monitor because it will have better performance:
1. Metrics monitor curently uses the same cached HTTP connection for every POST call while curl_krb_request will make different HTTP connection for each call
2. Metrics monitor can take advantage of the same cached HTTP connection for every POST call by using spnego_kerberos_auth
3. With spnego_kerberos_auth, metrics monitor will use cookie in memory while curl_krb_request uses cookie cache in disk
curl_krb_request can be better choice for alert monitors due to the nature of alerts - should be on different connection for each call.
- Qin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60403/#review179673
-----------------------------------------------------------
On June 23, 2017, 10:49 p.m., Qin Liu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60403/
> -----------------------------------------------------------
>
> (Updated June 23, 2017, 10:49 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan, Di Li, Dmytro Sen, Robert Levas, Sid Wagle, and Tim Thorpe.
>
>
> Bugs: AMBARI-17898
> https://issues.apache.org/jira/browse/AMBARI-17898
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics Monitor, a client of Ambari Metrics Collector, currently does not support Kerberos HTTP SPNEGO authentication.
> /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out:
> 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: http://metrics-collector:6188/ws/v1/timeline/metrics
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to server. HTTP Error 401: Authentication required
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ...
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini 7fe7397
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py d1429ed
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py 77b8c23
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/krberr.py PRE-CREATION
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py PRE-CREATION
> ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestEmitter.py 4056ae3
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json 51f541f
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/metainfo.xml 9031b46
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py f49d47d
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py b8c14f4
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2 b7dee50
>
>
> Diff: https://reviews.apache.org/r/60403/diff/1/
>
>
> Testing
> -------
>
> 1. Unit tests passed
> 2. manually tested with HDP/trunk
>
>
> Thanks,
>
> Qin Liu
>
>
Re: Review Request 60403: Add Kerberos HTTP SPNEGO authentication
support to Ambari Metrics Monitor
Posted by Qin Liu <qi...@gmail.com>.
> On July 5, 2017, 6:38 p.m., Robert Levas wrote:
> > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py
> > Lines 33 (patched)
> > <https://reviews.apache.org/r/60403/diff/1/?file=1761672#file1761672line33>
> >
> > Would it be possible to use `resource_management.libraries.functions.curl_krb_request.curl_krb_request` instead?
>
> Qin Liu wrote:
> Hi Robert,
> Thank you so much for your review! I just came back from vacation and will re-visit it ASAP.
>
> Qin Liu wrote:
> Hi Robert,
>
> Sorry for the delay!
>
> I think spnego_kerberos_auth is better choice than curl_krb_request for metrics monitor because it will have better performance:
> 1. Metrics monitor curently uses the same cached HTTP connection for every POST call while curl_krb_request will make different HTTP connection for each call
> 2. Metrics monitor can take advantage of the same cached HTTP connection for every POST call by using spnego_kerberos_auth
> 3. With spnego_kerberos_auth, metrics monitor will use cookie in memory while curl_krb_request uses cookie cache in disk
>
> curl_krb_request can be better choice for alert monitors due to the nature of alerts - should be on different connection for each call.
>
> Robert Levas wrote:
> Hey Qin... Thanks for the clarification. I will drop this issue.
>
> Sorry for the delay, I was out on vacation last week and am just getting caugh up.
Hi Robert,
Thank you so much!
- Qin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60403/#review179673
-----------------------------------------------------------
On June 23, 2017, 10:49 p.m., Qin Liu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60403/
> -----------------------------------------------------------
>
> (Updated June 23, 2017, 10:49 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan, Di Li, Dmytro Sen, Robert Levas, Sid Wagle, and Tim Thorpe.
>
>
> Bugs: AMBARI-17898
> https://issues.apache.org/jira/browse/AMBARI-17898
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But Ambari Metrics Monitor, a client of Ambari Metrics Collector, currently does not support Kerberos HTTP SPNEGO authentication.
> /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out:
> 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: http://metrics-collector:6188/ws/v1/timeline/metrics
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to server. HTTP Error 401: Authentication required
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ...
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini 7fe7397
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py d1429ed
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py 77b8c23
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/krberr.py PRE-CREATION
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py PRE-CREATION
> ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestEmitter.py 4056ae3
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json 51f541f
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/metainfo.xml 9031b46
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py f49d47d
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py b8c14f4
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2 b7dee50
>
>
> Diff: https://reviews.apache.org/r/60403/diff/1/
>
>
> Testing
> -------
>
> 1. Unit tests passed
> 2. manually tested with HDP/trunk
>
>
> Thanks,
>
> Qin Liu
>
>