You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Robert Levas <rl...@hortonworks.com> on 2014/11/27 18:47:13 UTC
Review Request 28519: Components should indicate Security State (via
ambari-agent)
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28519/
-----------------------------------------------------------
Review request for Ambari, Jaimin Jetly, Nate Cole, Robert Nettleton, and Tom Beerbower.
Bugs: AMBARI-8343
https://issues.apache.org/jira/browse/AMBARI-8343
Repository: ambari
Description
-------
In order to properly handle the automated installation or removal of a security infrastructure (like Kerberos) in the cluster, Ambari needs to know whether each component on the hosts of the cluster is properly _secured_ or not. This information may be compared with data on the Ambari server to help determine what steps should be taken to ensure the cluster is in the correct _secured_ state.
To do this, the current and desired component security state is maintained in the Ambari database. The Ambari server will update the desired state details according to whether the cluster is to be secured or not and whether the relevant service has enough metadata to be secured. If the desired and actual security state details do not match, the Ambari server will take the necessary steps to work towards synchronization.
In order for a component to indicate its security status, a new property needs to be returned in the `STATUS_COMMAND` response message (from the Ambari agent). This property should be named ‘securityState’ and should have one of the following values:
* `UNKNOWN` - Indicates that it is not known whether the service or component is secured or not
* `UNSECURED` - Indicates service or component is not or should not be secured
* `SECURED_KERBEROS` - Indicates component is or should be secured using Kerberos
* `ERROR` - Indicates the component is not secured due to an error condition
To properly set this state value, a call needs to be executed per component querying for its specific state. Due to the differences on how each component is secured and how it may be determined if security is setup what type is configured, and working is it properly, it is necessary for each component to have its own logic for determining this state. Therefore the ambari-agent process will need to call into the component’s configured (lifecycle) script and wait for its response - not unlike how it determines whether the component is up and running.
After the infrastructure is in place, each service definition needs to be updated to implement the new security status check function. The function should perform the following steps:
* Determine if security is enabled or disabled
** If disabled, return "UNSECURED"
** If enabled, determine what type of security is enabled
*** If Kerberos is configured
**** Perform tests (kinit?, ping KDC?) to determine if the configuration appears to be working
***** If working, return “SECURED_KERBEROS”
***** If not working, return “ERROR”
*** Else, return "UNKNOWN"
If no function is available, the Ambari agent should return “UNKNOWN”.
Diffs
-----
ambari-agent/src/main/python/ambari_agent/ActionQueue.py 4ecb822
ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 08dddae
ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 034dba3
ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 24ee259
Diff: https://reviews.apache.org/r/28519/diff/
Testing
-------
Updated the following unit tests:
* ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
* ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:06 min
[INFO] Finished at: 2014-11-27T15:35:18+00:00
[INFO] Final Memory: 80M/411M
[INFO] ------------------------------------------------------------------------
Thanks,
Robert Levas
Re: Review Request 28519: Components should indicate Security State
(via ambari-agent)
Posted by Robert Levas <rl...@hortonworks.com>.
> On Dec. 9, 2014, 11:10 a.m., Nate Cole wrote:
> > ambari-agent/src/main/python/ambari_agent/ActionQueue.py, line 334
> > <https://reviews.apache.org/r/28519/diff/2/?file=782692#file782692line334>
> >
> > Can a security check be part of a regular status check?
Looking at the existing satus check code, it appeares to want to return significantly different data. For example I think many implemenations throw a ComponentIsNotRunning exception if the component is down. This will hinder the abiity to obtain the security state of a component which is not related to the running state of a component.
> On Dec. 9, 2014, 11:10 a.m., Nate Cole wrote:
> > ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py, line 233
> > <https://reviews.apache.org/r/28519/diff/2/?file=782693#file782693line233>
> >
> > You are only checking one attribute out of the return dictionary. Can you just return the value in that case?
This will be fixed in the next patch.
> On Dec. 9, 2014, 11:10 a.m., Nate Cole wrote:
> > ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py, line 248
> > <https://reviews.apache.org/r/28519/diff/2/?file=782693#file782693line248>
> >
> > You may want to check for the structuredOut and securityState dictionary keys to actually be available. Otherwise you'll get nasty errors.
Adding checks and logging statements in next patch.
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28519/#review64380
-----------------------------------------------------------
On Dec. 9, 2014, 10:55 a.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28519/
> -----------------------------------------------------------
>
> (Updated Dec. 9, 2014, 10:55 a.m.)
>
>
> Review request for Ambari, Jaimin Jetly, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert Nettleton, and Tom Beerbower.
>
>
> Bugs: AMBARI-8343
> https://issues.apache.org/jira/browse/AMBARI-8343
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In order to properly handle the automated installation or removal of a security infrastructure (like Kerberos) in the cluster, Ambari needs to know whether each component on the hosts of the cluster is properly _secured_ or not. This information may be compared with data on the Ambari server to help determine what steps should be taken to ensure the cluster is in the correct _secured_ state.
>
> To do this, the current and desired component security state is maintained in the Ambari database. The Ambari server will update the desired state details according to whether the cluster is to be secured or not and whether the relevant service has enough metadata to be secured. If the desired and actual security state details do not match, the Ambari server will take the necessary steps to work towards synchronization.
>
> In order for a component to indicate its security status, a new property needs to be returned in the `STATUS_COMMAND` response message (from the Ambari agent). This property should be named ‘securityState’ and should have one of the following values:
>
> * `UNKNOWN` - Indicates that it is not known whether the service or component is secured or not
> * `UNSECURED` - Indicates service or component is not or should not be secured
> * `SECURED_KERBEROS` - Indicates component is or should be secured using Kerberos
> * `ERROR` - Indicates the component is not secured due to an error condition
>
> To properly set this state value, a call needs to be executed per component querying for its specific state. Due to the differences on how each component is secured and how it may be determined if security is setup what type is configured, and working is it properly, it is necessary for each component to have its own logic for determining this state. Therefore the ambari-agent process will need to call into the component’s configured (lifecycle) script and wait for its response - not unlike how it determines whether the component is up and running.
>
> After the infrastructure is in place, each service definition needs to be updated to implement the new security status check function. The function should perform the following steps:
>
> * Determine if security is enabled or disabled
> ** If disabled, return "UNSECURED"
> ** If enabled, determine what type of security is enabled
> *** If Kerberos is configured
> **** Perform tests (kinit?, ping KDC?) to determine if the configuration appears to be working
> ***** If working, return “SECURED_KERBEROS”
> ***** If not working, return “ERROR”
> *** Else, return "UNKNOWN"
>
> If no function is available, the Ambari agent should return “UNKNOWN”.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/ambari_agent/ActionQueue.py 4ecb822
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 08dddae
> ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 034dba3
> ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 24ee259
>
> Diff: https://reviews.apache.org/r/28519/diff/
>
>
> Testing
> -------
>
> Updated the following unit tests:
> * ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
> * ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 03:06 min
> [INFO] Finished at: 2014-11-27T15:35:18+00:00
> [INFO] Final Memory: 80M/411M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 28519: Components should indicate Security State
(via ambari-agent)
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28519/#review64380
-----------------------------------------------------------
ambari-agent/src/main/python/ambari_agent/ActionQueue.py
<https://reviews.apache.org/r/28519/#comment107058>
Can a security check be part of a regular status check?
ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
<https://reviews.apache.org/r/28519/#comment107061>
You are only checking one attribute out of the return dictionary. Can you just return the value in that case?
ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
<https://reviews.apache.org/r/28519/#comment107057>
You may want to check for the structuredOut and securityState dictionary keys to actually be available. Otherwise you'll get nasty errors.
- Nate Cole
On Dec. 9, 2014, 10:55 a.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28519/
> -----------------------------------------------------------
>
> (Updated Dec. 9, 2014, 10:55 a.m.)
>
>
> Review request for Ambari, Jaimin Jetly, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert Nettleton, and Tom Beerbower.
>
>
> Bugs: AMBARI-8343
> https://issues.apache.org/jira/browse/AMBARI-8343
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In order to properly handle the automated installation or removal of a security infrastructure (like Kerberos) in the cluster, Ambari needs to know whether each component on the hosts of the cluster is properly _secured_ or not. This information may be compared with data on the Ambari server to help determine what steps should be taken to ensure the cluster is in the correct _secured_ state.
>
> To do this, the current and desired component security state is maintained in the Ambari database. The Ambari server will update the desired state details according to whether the cluster is to be secured or not and whether the relevant service has enough metadata to be secured. If the desired and actual security state details do not match, the Ambari server will take the necessary steps to work towards synchronization.
>
> In order for a component to indicate its security status, a new property needs to be returned in the `STATUS_COMMAND` response message (from the Ambari agent). This property should be named ‘securityState’ and should have one of the following values:
>
> * `UNKNOWN` - Indicates that it is not known whether the service or component is secured or not
> * `UNSECURED` - Indicates service or component is not or should not be secured
> * `SECURED_KERBEROS` - Indicates component is or should be secured using Kerberos
> * `ERROR` - Indicates the component is not secured due to an error condition
>
> To properly set this state value, a call needs to be executed per component querying for its specific state. Due to the differences on how each component is secured and how it may be determined if security is setup what type is configured, and working is it properly, it is necessary for each component to have its own logic for determining this state. Therefore the ambari-agent process will need to call into the component’s configured (lifecycle) script and wait for its response - not unlike how it determines whether the component is up and running.
>
> After the infrastructure is in place, each service definition needs to be updated to implement the new security status check function. The function should perform the following steps:
>
> * Determine if security is enabled or disabled
> ** If disabled, return "UNSECURED"
> ** If enabled, determine what type of security is enabled
> *** If Kerberos is configured
> **** Perform tests (kinit?, ping KDC?) to determine if the configuration appears to be working
> ***** If working, return “SECURED_KERBEROS”
> ***** If not working, return “ERROR”
> *** Else, return "UNKNOWN"
>
> If no function is available, the Ambari agent should return “UNKNOWN”.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/ambari_agent/ActionQueue.py 4ecb822
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 08dddae
> ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 034dba3
> ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 24ee259
>
> Diff: https://reviews.apache.org/r/28519/diff/
>
>
> Testing
> -------
>
> Updated the following unit tests:
> * ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
> * ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 03:06 min
> [INFO] Finished at: 2014-11-27T15:35:18+00:00
> [INFO] Final Memory: 80M/411M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 28519: Components should indicate Security State
(via ambari-agent)
Posted by Jayush Luniya <jl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28519/#review64524
-----------------------------------------------------------
ambari-agent/src/main/python/ambari_agent/ActionQueue.py
<https://reviews.apache.org/r/28519/#comment107286>
Check if key securityState exists
- Jayush Luniya
On Dec. 9, 2014, 3:55 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28519/
> -----------------------------------------------------------
>
> (Updated Dec. 9, 2014, 3:55 p.m.)
>
>
> Review request for Ambari, Jaimin Jetly, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert Nettleton, and Tom Beerbower.
>
>
> Bugs: AMBARI-8343
> https://issues.apache.org/jira/browse/AMBARI-8343
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In order to properly handle the automated installation or removal of a security infrastructure (like Kerberos) in the cluster, Ambari needs to know whether each component on the hosts of the cluster is properly _secured_ or not. This information may be compared with data on the Ambari server to help determine what steps should be taken to ensure the cluster is in the correct _secured_ state.
>
> To do this, the current and desired component security state is maintained in the Ambari database. The Ambari server will update the desired state details according to whether the cluster is to be secured or not and whether the relevant service has enough metadata to be secured. If the desired and actual security state details do not match, the Ambari server will take the necessary steps to work towards synchronization.
>
> In order for a component to indicate its security status, a new property needs to be returned in the `STATUS_COMMAND` response message (from the Ambari agent). This property should be named ‘securityState’ and should have one of the following values:
>
> * `UNKNOWN` - Indicates that it is not known whether the service or component is secured or not
> * `UNSECURED` - Indicates service or component is not or should not be secured
> * `SECURED_KERBEROS` - Indicates component is or should be secured using Kerberos
> * `ERROR` - Indicates the component is not secured due to an error condition
>
> To properly set this state value, a call needs to be executed per component querying for its specific state. Due to the differences on how each component is secured and how it may be determined if security is setup what type is configured, and working is it properly, it is necessary for each component to have its own logic for determining this state. Therefore the ambari-agent process will need to call into the component’s configured (lifecycle) script and wait for its response - not unlike how it determines whether the component is up and running.
>
> After the infrastructure is in place, each service definition needs to be updated to implement the new security status check function. The function should perform the following steps:
>
> * Determine if security is enabled or disabled
> ** If disabled, return "UNSECURED"
> ** If enabled, determine what type of security is enabled
> *** If Kerberos is configured
> **** Perform tests (kinit?, ping KDC?) to determine if the configuration appears to be working
> ***** If working, return “SECURED_KERBEROS”
> ***** If not working, return “ERROR”
> *** Else, return "UNKNOWN"
>
> If no function is available, the Ambari agent should return “UNKNOWN”.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/ambari_agent/ActionQueue.py 4ecb822
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 08dddae
> ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 034dba3
> ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 24ee259
>
> Diff: https://reviews.apache.org/r/28519/diff/
>
>
> Testing
> -------
>
> Updated the following unit tests:
> * ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
> * ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 03:06 min
> [INFO] Finished at: 2014-11-27T15:35:18+00:00
> [INFO] Final Memory: 80M/411M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 28519: Components should indicate Security State
(via ambari-agent)
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28519/#review64825
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On Dec. 10, 2014, 7:11 a.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28519/
> -----------------------------------------------------------
>
> (Updated Dec. 10, 2014, 7:11 a.m.)
>
>
> Review request for Ambari, Jaimin Jetly, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert Nettleton, and Tom Beerbower.
>
>
> Bugs: AMBARI-8343
> https://issues.apache.org/jira/browse/AMBARI-8343
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In order to properly handle the automated installation or removal of a security infrastructure (like Kerberos) in the cluster, Ambari needs to know whether each component on the hosts of the cluster is properly _secured_ or not. This information may be compared with data on the Ambari server to help determine what steps should be taken to ensure the cluster is in the correct _secured_ state.
>
> To do this, the current and desired component security state is maintained in the Ambari database. The Ambari server will update the desired state details according to whether the cluster is to be secured or not and whether the relevant service has enough metadata to be secured. If the desired and actual security state details do not match, the Ambari server will take the necessary steps to work towards synchronization.
>
> In order for a component to indicate its security status, a new property needs to be returned in the `STATUS_COMMAND` response message (from the Ambari agent). This property should be named ‘securityState’ and should have one of the following values:
>
> * `UNKNOWN` - Indicates that it is not known whether the service or component is secured or not
> * `UNSECURED` - Indicates service or component is not or should not be secured
> * `SECURED_KERBEROS` - Indicates component is or should be secured using Kerberos
> * `ERROR` - Indicates the component is not secured due to an error condition
>
> To properly set this state value, a call needs to be executed per component querying for its specific state. Due to the differences on how each component is secured and how it may be determined if security is setup what type is configured, and working is it properly, it is necessary for each component to have its own logic for determining this state. Therefore the ambari-agent process will need to call into the component’s configured (lifecycle) script and wait for its response - not unlike how it determines whether the component is up and running.
>
> After the infrastructure is in place, each service definition needs to be updated to implement the new security status check function. The function should perform the following steps:
>
> * Determine if security is enabled or disabled
> ** If disabled, return "UNSECURED"
> ** If enabled, determine what type of security is enabled
> *** If Kerberos is configured
> **** Perform tests (kinit?, ping KDC?) to determine if the configuration appears to be working
> ***** If working, return “SECURED_KERBEROS”
> ***** If not working, return “ERROR”
> *** Else, return "UNKNOWN"
>
> If no function is available, the Ambari agent should return “UNKNOWN”.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/ambari_agent/ActionQueue.py fbde26f
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 08dddae
> ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 52b586b
> ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 24ee259
>
> Diff: https://reviews.apache.org/r/28519/diff/
>
>
> Testing
> -------
>
> Updated the following unit tests:
> * ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
> * ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 03:06 min
> [INFO] Finished at: 2014-11-27T15:35:18+00:00
> [INFO] Final Memory: 80M/411M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 28519: Components should indicate Security State
(via ambari-agent)
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28519/#review64837
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On Dec. 10, 2014, 7:11 a.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28519/
> -----------------------------------------------------------
>
> (Updated Dec. 10, 2014, 7:11 a.m.)
>
>
> Review request for Ambari, Jaimin Jetly, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert Nettleton, and Tom Beerbower.
>
>
> Bugs: AMBARI-8343
> https://issues.apache.org/jira/browse/AMBARI-8343
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In order to properly handle the automated installation or removal of a security infrastructure (like Kerberos) in the cluster, Ambari needs to know whether each component on the hosts of the cluster is properly _secured_ or not. This information may be compared with data on the Ambari server to help determine what steps should be taken to ensure the cluster is in the correct _secured_ state.
>
> To do this, the current and desired component security state is maintained in the Ambari database. The Ambari server will update the desired state details according to whether the cluster is to be secured or not and whether the relevant service has enough metadata to be secured. If the desired and actual security state details do not match, the Ambari server will take the necessary steps to work towards synchronization.
>
> In order for a component to indicate its security status, a new property needs to be returned in the `STATUS_COMMAND` response message (from the Ambari agent). This property should be named ‘securityState’ and should have one of the following values:
>
> * `UNKNOWN` - Indicates that it is not known whether the service or component is secured or not
> * `UNSECURED` - Indicates service or component is not or should not be secured
> * `SECURED_KERBEROS` - Indicates component is or should be secured using Kerberos
> * `ERROR` - Indicates the component is not secured due to an error condition
>
> To properly set this state value, a call needs to be executed per component querying for its specific state. Due to the differences on how each component is secured and how it may be determined if security is setup what type is configured, and working is it properly, it is necessary for each component to have its own logic for determining this state. Therefore the ambari-agent process will need to call into the component’s configured (lifecycle) script and wait for its response - not unlike how it determines whether the component is up and running.
>
> After the infrastructure is in place, each service definition needs to be updated to implement the new security status check function. The function should perform the following steps:
>
> * Determine if security is enabled or disabled
> ** If disabled, return "UNSECURED"
> ** If enabled, determine what type of security is enabled
> *** If Kerberos is configured
> **** Perform tests (kinit?, ping KDC?) to determine if the configuration appears to be working
> ***** If working, return “SECURED_KERBEROS”
> ***** If not working, return “ERROR”
> *** Else, return "UNKNOWN"
>
> If no function is available, the Ambari agent should return “UNKNOWN”.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/ambari_agent/ActionQueue.py fbde26f
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 08dddae
> ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 52b586b
> ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 24ee259
>
> Diff: https://reviews.apache.org/r/28519/diff/
>
>
> Testing
> -------
>
> Updated the following unit tests:
> * ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
> * ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 03:06 min
> [INFO] Finished at: 2014-11-27T15:35:18+00:00
> [INFO] Final Memory: 80M/411M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 28519: Components should indicate Security State
(via ambari-agent)
Posted by Jayush Luniya <jl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28519/#review64871
-----------------------------------------------------------
Ship it!
Ship It!
- Jayush Luniya
On Dec. 10, 2014, 12:11 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28519/
> -----------------------------------------------------------
>
> (Updated Dec. 10, 2014, 12:11 p.m.)
>
>
> Review request for Ambari, Jaimin Jetly, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert Nettleton, and Tom Beerbower.
>
>
> Bugs: AMBARI-8343
> https://issues.apache.org/jira/browse/AMBARI-8343
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In order to properly handle the automated installation or removal of a security infrastructure (like Kerberos) in the cluster, Ambari needs to know whether each component on the hosts of the cluster is properly _secured_ or not. This information may be compared with data on the Ambari server to help determine what steps should be taken to ensure the cluster is in the correct _secured_ state.
>
> To do this, the current and desired component security state is maintained in the Ambari database. The Ambari server will update the desired state details according to whether the cluster is to be secured or not and whether the relevant service has enough metadata to be secured. If the desired and actual security state details do not match, the Ambari server will take the necessary steps to work towards synchronization.
>
> In order for a component to indicate its security status, a new property needs to be returned in the `STATUS_COMMAND` response message (from the Ambari agent). This property should be named ‘securityState’ and should have one of the following values:
>
> * `UNKNOWN` - Indicates that it is not known whether the service or component is secured or not
> * `UNSECURED` - Indicates service or component is not or should not be secured
> * `SECURED_KERBEROS` - Indicates component is or should be secured using Kerberos
> * `ERROR` - Indicates the component is not secured due to an error condition
>
> To properly set this state value, a call needs to be executed per component querying for its specific state. Due to the differences on how each component is secured and how it may be determined if security is setup what type is configured, and working is it properly, it is necessary for each component to have its own logic for determining this state. Therefore the ambari-agent process will need to call into the component’s configured (lifecycle) script and wait for its response - not unlike how it determines whether the component is up and running.
>
> After the infrastructure is in place, each service definition needs to be updated to implement the new security status check function. The function should perform the following steps:
>
> * Determine if security is enabled or disabled
> ** If disabled, return "UNSECURED"
> ** If enabled, determine what type of security is enabled
> *** If Kerberos is configured
> **** Perform tests (kinit?, ping KDC?) to determine if the configuration appears to be working
> ***** If working, return “SECURED_KERBEROS”
> ***** If not working, return “ERROR”
> *** Else, return "UNKNOWN"
>
> If no function is available, the Ambari agent should return “UNKNOWN”.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/ambari_agent/ActionQueue.py fbde26f
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 08dddae
> ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 52b586b
> ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 24ee259
>
> Diff: https://reviews.apache.org/r/28519/diff/
>
>
> Testing
> -------
>
> Updated the following unit tests:
> * ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
> * ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 03:06 min
> [INFO] Finished at: 2014-11-27T15:35:18+00:00
> [INFO] Final Memory: 80M/411M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 28519: Components should indicate Security State
(via ambari-agent)
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28519/
-----------------------------------------------------------
(Updated Dec. 10, 2014, 7:11 a.m.)
Review request for Ambari, Jaimin Jetly, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert Nettleton, and Tom Beerbower.
Changes
-------
Updates addressing comments.
Bugs: AMBARI-8343
https://issues.apache.org/jira/browse/AMBARI-8343
Repository: ambari
Description
-------
In order to properly handle the automated installation or removal of a security infrastructure (like Kerberos) in the cluster, Ambari needs to know whether each component on the hosts of the cluster is properly _secured_ or not. This information may be compared with data on the Ambari server to help determine what steps should be taken to ensure the cluster is in the correct _secured_ state.
To do this, the current and desired component security state is maintained in the Ambari database. The Ambari server will update the desired state details according to whether the cluster is to be secured or not and whether the relevant service has enough metadata to be secured. If the desired and actual security state details do not match, the Ambari server will take the necessary steps to work towards synchronization.
In order for a component to indicate its security status, a new property needs to be returned in the `STATUS_COMMAND` response message (from the Ambari agent). This property should be named ‘securityState’ and should have one of the following values:
* `UNKNOWN` - Indicates that it is not known whether the service or component is secured or not
* `UNSECURED` - Indicates service or component is not or should not be secured
* `SECURED_KERBEROS` - Indicates component is or should be secured using Kerberos
* `ERROR` - Indicates the component is not secured due to an error condition
To properly set this state value, a call needs to be executed per component querying for its specific state. Due to the differences on how each component is secured and how it may be determined if security is setup what type is configured, and working is it properly, it is necessary for each component to have its own logic for determining this state. Therefore the ambari-agent process will need to call into the component’s configured (lifecycle) script and wait for its response - not unlike how it determines whether the component is up and running.
After the infrastructure is in place, each service definition needs to be updated to implement the new security status check function. The function should perform the following steps:
* Determine if security is enabled or disabled
** If disabled, return "UNSECURED"
** If enabled, determine what type of security is enabled
*** If Kerberos is configured
**** Perform tests (kinit?, ping KDC?) to determine if the configuration appears to be working
***** If working, return “SECURED_KERBEROS”
***** If not working, return “ERROR”
*** Else, return "UNKNOWN"
If no function is available, the Ambari agent should return “UNKNOWN”.
Diffs (updated)
-----
ambari-agent/src/main/python/ambari_agent/ActionQueue.py fbde26f
ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 08dddae
ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 52b586b
ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 24ee259
Diff: https://reviews.apache.org/r/28519/diff/
Testing
-------
Updated the following unit tests:
* ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
* ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:06 min
[INFO] Finished at: 2014-11-27T15:35:18+00:00
[INFO] Final Memory: 80M/411M
[INFO] ------------------------------------------------------------------------
Thanks,
Robert Levas
Re: Review Request 28519: Components should indicate Security State
(via ambari-agent)
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28519/
-----------------------------------------------------------
(Updated Dec. 9, 2014, 10:55 a.m.)
Review request for Ambari, Jaimin Jetly, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert Nettleton, and Tom Beerbower.
Changes
-------
Requesting more reviewers knowlegable of agent-side code.
Bugs: AMBARI-8343
https://issues.apache.org/jira/browse/AMBARI-8343
Repository: ambari
Description
-------
In order to properly handle the automated installation or removal of a security infrastructure (like Kerberos) in the cluster, Ambari needs to know whether each component on the hosts of the cluster is properly _secured_ or not. This information may be compared with data on the Ambari server to help determine what steps should be taken to ensure the cluster is in the correct _secured_ state.
To do this, the current and desired component security state is maintained in the Ambari database. The Ambari server will update the desired state details according to whether the cluster is to be secured or not and whether the relevant service has enough metadata to be secured. If the desired and actual security state details do not match, the Ambari server will take the necessary steps to work towards synchronization.
In order for a component to indicate its security status, a new property needs to be returned in the `STATUS_COMMAND` response message (from the Ambari agent). This property should be named ‘securityState’ and should have one of the following values:
* `UNKNOWN` - Indicates that it is not known whether the service or component is secured or not
* `UNSECURED` - Indicates service or component is not or should not be secured
* `SECURED_KERBEROS` - Indicates component is or should be secured using Kerberos
* `ERROR` - Indicates the component is not secured due to an error condition
To properly set this state value, a call needs to be executed per component querying for its specific state. Due to the differences on how each component is secured and how it may be determined if security is setup what type is configured, and working is it properly, it is necessary for each component to have its own logic for determining this state. Therefore the ambari-agent process will need to call into the component’s configured (lifecycle) script and wait for its response - not unlike how it determines whether the component is up and running.
After the infrastructure is in place, each service definition needs to be updated to implement the new security status check function. The function should perform the following steps:
* Determine if security is enabled or disabled
** If disabled, return "UNSECURED"
** If enabled, determine what type of security is enabled
*** If Kerberos is configured
**** Perform tests (kinit?, ping KDC?) to determine if the configuration appears to be working
***** If working, return “SECURED_KERBEROS”
***** If not working, return “ERROR”
*** Else, return "UNKNOWN"
If no function is available, the Ambari agent should return “UNKNOWN”.
Diffs
-----
ambari-agent/src/main/python/ambari_agent/ActionQueue.py 4ecb822
ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 08dddae
ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 034dba3
ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 24ee259
Diff: https://reviews.apache.org/r/28519/diff/
Testing
-------
Updated the following unit tests:
* ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
* ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:06 min
[INFO] Finished at: 2014-11-27T15:35:18+00:00
[INFO] Final Memory: 80M/411M
[INFO] ------------------------------------------------------------------------
Thanks,
Robert Levas
Re: Review Request 28519: Components should indicate Security State
(via ambari-agent)
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28519/
-----------------------------------------------------------
(Updated Dec. 4, 2014, 1:48 p.m.)
Review request for Ambari, Jaimin Jetly, Nate Cole, Robert Nettleton, and Tom Beerbower.
Changes
-------
Updated unit tests to test for failure cases.
Bugs: AMBARI-8343
https://issues.apache.org/jira/browse/AMBARI-8343
Repository: ambari
Description
-------
In order to properly handle the automated installation or removal of a security infrastructure (like Kerberos) in the cluster, Ambari needs to know whether each component on the hosts of the cluster is properly _secured_ or not. This information may be compared with data on the Ambari server to help determine what steps should be taken to ensure the cluster is in the correct _secured_ state.
To do this, the current and desired component security state is maintained in the Ambari database. The Ambari server will update the desired state details according to whether the cluster is to be secured or not and whether the relevant service has enough metadata to be secured. If the desired and actual security state details do not match, the Ambari server will take the necessary steps to work towards synchronization.
In order for a component to indicate its security status, a new property needs to be returned in the `STATUS_COMMAND` response message (from the Ambari agent). This property should be named ‘securityState’ and should have one of the following values:
* `UNKNOWN` - Indicates that it is not known whether the service or component is secured or not
* `UNSECURED` - Indicates service or component is not or should not be secured
* `SECURED_KERBEROS` - Indicates component is or should be secured using Kerberos
* `ERROR` - Indicates the component is not secured due to an error condition
To properly set this state value, a call needs to be executed per component querying for its specific state. Due to the differences on how each component is secured and how it may be determined if security is setup what type is configured, and working is it properly, it is necessary for each component to have its own logic for determining this state. Therefore the ambari-agent process will need to call into the component’s configured (lifecycle) script and wait for its response - not unlike how it determines whether the component is up and running.
After the infrastructure is in place, each service definition needs to be updated to implement the new security status check function. The function should perform the following steps:
* Determine if security is enabled or disabled
** If disabled, return "UNSECURED"
** If enabled, determine what type of security is enabled
*** If Kerberos is configured
**** Perform tests (kinit?, ping KDC?) to determine if the configuration appears to be working
***** If working, return “SECURED_KERBEROS”
***** If not working, return “ERROR”
*** Else, return "UNKNOWN"
If no function is available, the Ambari agent should return “UNKNOWN”.
Diffs (updated)
-----
ambari-agent/src/main/python/ambari_agent/ActionQueue.py 4ecb822
ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 08dddae
ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 034dba3
ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 24ee259
Diff: https://reviews.apache.org/r/28519/diff/
Testing
-------
Updated the following unit tests:
* ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
* ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:06 min
[INFO] Finished at: 2014-11-27T15:35:18+00:00
[INFO] Final Memory: 80M/411M
[INFO] ------------------------------------------------------------------------
Thanks,
Robert Levas
Re: Review Request 28519: Components should indicate Security State
(via ambari-agent)
Posted by Robert Nettleton <rn...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28519/#review63365
-----------------------------------------------------------
Ship it!
Patch looks good to me, other than the minor issue I've filed regarding a new unit test to be added.
ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
<https://reviews.apache.org/r/28519/#comment105581>
I'd recommend adding another unit test to verify that the default "securityState" is "UNKNOWN" in the event of an error during the attempt to run the status command.
- Robert Nettleton
On Nov. 27, 2014, 5:47 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28519/
> -----------------------------------------------------------
>
> (Updated Nov. 27, 2014, 5:47 p.m.)
>
>
> Review request for Ambari, Jaimin Jetly, Nate Cole, Robert Nettleton, and Tom Beerbower.
>
>
> Bugs: AMBARI-8343
> https://issues.apache.org/jira/browse/AMBARI-8343
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In order to properly handle the automated installation or removal of a security infrastructure (like Kerberos) in the cluster, Ambari needs to know whether each component on the hosts of the cluster is properly _secured_ or not. This information may be compared with data on the Ambari server to help determine what steps should be taken to ensure the cluster is in the correct _secured_ state.
>
> To do this, the current and desired component security state is maintained in the Ambari database. The Ambari server will update the desired state details according to whether the cluster is to be secured or not and whether the relevant service has enough metadata to be secured. If the desired and actual security state details do not match, the Ambari server will take the necessary steps to work towards synchronization.
>
> In order for a component to indicate its security status, a new property needs to be returned in the `STATUS_COMMAND` response message (from the Ambari agent). This property should be named ‘securityState’ and should have one of the following values:
>
> * `UNKNOWN` - Indicates that it is not known whether the service or component is secured or not
> * `UNSECURED` - Indicates service or component is not or should not be secured
> * `SECURED_KERBEROS` - Indicates component is or should be secured using Kerberos
> * `ERROR` - Indicates the component is not secured due to an error condition
>
> To properly set this state value, a call needs to be executed per component querying for its specific state. Due to the differences on how each component is secured and how it may be determined if security is setup what type is configured, and working is it properly, it is necessary for each component to have its own logic for determining this state. Therefore the ambari-agent process will need to call into the component’s configured (lifecycle) script and wait for its response - not unlike how it determines whether the component is up and running.
>
> After the infrastructure is in place, each service definition needs to be updated to implement the new security status check function. The function should perform the following steps:
>
> * Determine if security is enabled or disabled
> ** If disabled, return "UNSECURED"
> ** If enabled, determine what type of security is enabled
> *** If Kerberos is configured
> **** Perform tests (kinit?, ping KDC?) to determine if the configuration appears to be working
> ***** If working, return “SECURED_KERBEROS”
> ***** If not working, return “ERROR”
> *** Else, return "UNKNOWN"
>
> If no function is available, the Ambari agent should return “UNKNOWN”.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/ambari_agent/ActionQueue.py 4ecb822
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 08dddae
> ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 034dba3
> ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 24ee259
>
> Diff: https://reviews.apache.org/r/28519/diff/
>
>
> Testing
> -------
>
> Updated the following unit tests:
> * ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
> * ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 03:06 min
> [INFO] Finished at: 2014-11-27T15:35:18+00:00
> [INFO] Final Memory: 80M/411M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 28519: Components should indicate Security State
(via ambari-agent)
Posted by Tom Beerbower <tb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28519/#review63378
-----------------------------------------------------------
Ship it!
Ship It!
- Tom Beerbower
On Nov. 27, 2014, 5:47 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28519/
> -----------------------------------------------------------
>
> (Updated Nov. 27, 2014, 5:47 p.m.)
>
>
> Review request for Ambari, Jaimin Jetly, Nate Cole, Robert Nettleton, and Tom Beerbower.
>
>
> Bugs: AMBARI-8343
> https://issues.apache.org/jira/browse/AMBARI-8343
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In order to properly handle the automated installation or removal of a security infrastructure (like Kerberos) in the cluster, Ambari needs to know whether each component on the hosts of the cluster is properly _secured_ or not. This information may be compared with data on the Ambari server to help determine what steps should be taken to ensure the cluster is in the correct _secured_ state.
>
> To do this, the current and desired component security state is maintained in the Ambari database. The Ambari server will update the desired state details according to whether the cluster is to be secured or not and whether the relevant service has enough metadata to be secured. If the desired and actual security state details do not match, the Ambari server will take the necessary steps to work towards synchronization.
>
> In order for a component to indicate its security status, a new property needs to be returned in the `STATUS_COMMAND` response message (from the Ambari agent). This property should be named ‘securityState’ and should have one of the following values:
>
> * `UNKNOWN` - Indicates that it is not known whether the service or component is secured or not
> * `UNSECURED` - Indicates service or component is not or should not be secured
> * `SECURED_KERBEROS` - Indicates component is or should be secured using Kerberos
> * `ERROR` - Indicates the component is not secured due to an error condition
>
> To properly set this state value, a call needs to be executed per component querying for its specific state. Due to the differences on how each component is secured and how it may be determined if security is setup what type is configured, and working is it properly, it is necessary for each component to have its own logic for determining this state. Therefore the ambari-agent process will need to call into the component’s configured (lifecycle) script and wait for its response - not unlike how it determines whether the component is up and running.
>
> After the infrastructure is in place, each service definition needs to be updated to implement the new security status check function. The function should perform the following steps:
>
> * Determine if security is enabled or disabled
> ** If disabled, return "UNSECURED"
> ** If enabled, determine what type of security is enabled
> *** If Kerberos is configured
> **** Perform tests (kinit?, ping KDC?) to determine if the configuration appears to be working
> ***** If working, return “SECURED_KERBEROS”
> ***** If not working, return “ERROR”
> *** Else, return "UNKNOWN"
>
> If no function is available, the Ambari agent should return “UNKNOWN”.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/ambari_agent/ActionQueue.py 4ecb822
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 08dddae
> ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 034dba3
> ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 24ee259
>
> Diff: https://reviews.apache.org/r/28519/diff/
>
>
> Testing
> -------
>
> Updated the following unit tests:
> * ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
> * ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 03:06 min
> [INFO] Finished at: 2014-11-27T15:35:18+00:00
> [INFO] Final Memory: 80M/411M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Robert Levas
>
>