You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@ambari.apache.org by Vitalyi Brodetskyi <vb...@hortonworks.com> on 2016/01/14 15:33:46 UTC

Review Request 42306: Kerberos specifical properties reverted during secured cluster deploy via blueprint

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42306/
-----------------------------------------------------------

Review request for Ambari, Robert Levas and Robert Nettleton.

Bugs: AMBARI-14668
https://issues.apache.org/jira/browse/AMBARI-14668

Repository: ambari

Description
-------

1) Deploy cluster with HDFS,YARN,Oozie, Storm,Zookeeper.
2) Enable security.
3) Customize some properties: hadoop.proxyuser.yarn.hosts, hadoop.proxyuser.HTTP.groups, oozie.authentication.type, yarn.timeline-service.http-authentication.type.
4) Export blueprint config. Check that customized properties are correct in exported config.
5) Deploy kerberized cluster via blueprints.
6) Check that after deploy customized properties are the same as they were set during customization.
Result: customized properties were reverted. Looks like during BP deploy, we are just updating all kerberos properties with values from kerberos.json from related service

Diffs
-----

ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 739cd43
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java b6686b4
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java 0b586c9
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java c885ca7
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java 9390b3a

Diff: https://reviews.apache.org/r/42306/diff/

Testing
-------

mvn clean test

Thanks,

Vitalyi Brodetskyi

Re: Review Request 42306: Kerberos specifical properties reverted during secured cluster deploy via blueprint

Posted by Robert Levas <rl...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42306/#review114482
-----------------------------------------------------------

Ship it!



ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java (line 125)
<https://reviews.apache.org/r/42306/#comment175335>

    I think that this may be a problem in the event a user puts default values in the BP as there is no good way to tell if the user is customizing a value or blindly copying default values. 
    
    Maybe it might be beneficial to compare the propery value from the BP with the default fromt he stack. If they are different then the user is customizing the value and therefore the Kerberos logic should not change it.   However, then there is no way to tell if the user wants to value to remain in it's default state. 
    
    @rnettleton, any suggestions for this?


- Robert Levas


On Jan. 14, 2016, 9:59 a.m., Vitalyi Brodetskyi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42306/
> -----------------------------------------------------------
> 
> (Updated Jan. 14, 2016, 9:59 a.m.)
> 
> 
> Review request for Ambari, Robert Levas and Robert Nettleton.
> 
> 
> Bugs: AMBARI-14668
>     https://issues.apache.org/jira/browse/AMBARI-14668
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 1) Deploy cluster with HDFS,YARN,Oozie, Storm,Zookeeper.
> 2) Enable security.
> 3) Customize some properties: hadoop.proxyuser.yarn.hosts, hadoop.proxyuser.HTTP.groups, oozie.authentication.type, yarn.timeline-service.http-authentication.type.
> 4) Export blueprint config. Check that customized properties are correct in exported config.
> 5) Deploy kerberized cluster via blueprints.
> 6) Check that after deploy customized properties are the same as they were set during customization.
> Result: customized properties were reverted. Looks like during BP deploy, we are just updating all kerberos properties with values from kerberos.json from related service
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 739cd43 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java c885ca7 
>   ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java 9390b3a 
> 
> Diff: https://reviews.apache.org/r/42306/diff/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 42306: Kerberos specifical properties reverted during secured cluster deploy via blueprint

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.

(Updated Січ. 15, 2016, 8:09 після полудня)

Review request for Ambari, Robert Levas and Robert Nettleton.

Bugs: AMBARI-14668
https://issues.apache.org/jira/browse/AMBARI-14668

Repository: ambari

Description
-------

Diffs (updated)
-----

ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 29099ec
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java 16d983a
ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterConfigurationRequestTest.java 96abe9d

Diff: https://reviews.apache.org/r/42306/diff/

Testing
-------

mvn clean test

Thanks,

Vitalyi Brodetskyi

Re: Review Request 42306: Kerberos specifical properties reverted during secured cluster deploy via blueprint

Posted by Robert Levas <rl...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42306/#review114719
-----------------------------------------------------------

Ship it!


Ship It!

- Robert Levas


On Jan. 15, 2016, 12:28 p.m., Vitalyi Brodetskyi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42306/
> -----------------------------------------------------------
> 
> (Updated Jan. 15, 2016, 12:28 p.m.)
> 
> 
> Review request for Ambari, Robert Levas and Robert Nettleton.
> 
> 
> Bugs: AMBARI-14668
>     https://issues.apache.org/jira/browse/AMBARI-14668
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 1) Deploy cluster with HDFS,YARN,Oozie, Storm,Zookeeper.
> 2) Enable security.
> 3) Customize some properties: hadoop.proxyuser.yarn.hosts, hadoop.proxyuser.HTTP.groups, oozie.authentication.type, yarn.timeline-service.http-authentication.type.
> 4) Export blueprint config. Check that customized properties are correct in exported config.
> 5) Deploy kerberized cluster via blueprints.
> 6) Check that after deploy customized properties are the same as they were set during customization.
> Result: customized properties were reverted. Looks like during BP deploy, we are just updating all kerberos properties with values from kerberos.json from related service
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 29099ec 
>   ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java 16d983a 
>   ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterConfigurationRequestTest.java 96abe9d 
> 
> Diff: https://reviews.apache.org/r/42306/diff/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 42306: Kerberos specifical properties reverted during secured cluster deploy via blueprint

Posted by Robert Nettleton <rn...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42306/#review114739
-----------------------------------------------------------

Ship it!


Ship It!

- Robert Nettleton


On Jan. 15, 2016, 5:28 p.m., Vitalyi Brodetskyi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42306/
> -----------------------------------------------------------
> 
> (Updated Jan. 15, 2016, 5:28 p.m.)
> 
> 
> Review request for Ambari, Robert Levas and Robert Nettleton.
> 
> 
> Bugs: AMBARI-14668
>     https://issues.apache.org/jira/browse/AMBARI-14668
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 1) Deploy cluster with HDFS,YARN,Oozie, Storm,Zookeeper.
> 2) Enable security.
> 3) Customize some properties: hadoop.proxyuser.yarn.hosts, hadoop.proxyuser.HTTP.groups, oozie.authentication.type, yarn.timeline-service.http-authentication.type.
> 4) Export blueprint config. Check that customized properties are correct in exported config.
> 5) Deploy kerberized cluster via blueprints.
> 6) Check that after deploy customized properties are the same as they were set during customization.
> Result: customized properties were reverted. Looks like during BP deploy, we are just updating all kerberos properties with values from kerberos.json from related service
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 29099ec 
>   ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java 16d983a 
>   ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterConfigurationRequestTest.java 96abe9d 
> 
> Diff: https://reviews.apache.org/r/42306/diff/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 42306: Kerberos specifical properties reverted during secured cluster deploy via blueprint

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.

(Updated Січ. 15, 2016, 5:28 після полудня)

Review request for Ambari, Robert Levas and Robert Nettleton.

Bugs: AMBARI-14668
https://issues.apache.org/jira/browse/AMBARI-14668

Repository: ambari

Description
-------

Diffs (updated)
-----

Diff: https://reviews.apache.org/r/42306/diff/

Testing
-------

mvn clean test

Thanks,

Vitalyi Brodetskyi

Re: Review Request 42306: Kerberos specifical properties reverted during secured cluster deploy via blueprint

Posted by Robert Nettleton <rn...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42306/#review114545
-----------------------------------------------------------

Ship it!


Ship It!

- Robert Nettleton


On Jan. 14, 2016, 6:54 p.m., Vitalyi Brodetskyi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42306/
> -----------------------------------------------------------
> 
> (Updated Jan. 14, 2016, 6:54 p.m.)
> 
> 
> Review request for Ambari, Robert Levas and Robert Nettleton.
> 
> 
> Bugs: AMBARI-14668
>     https://issues.apache.org/jira/browse/AMBARI-14668
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 1) Deploy cluster with HDFS,YARN,Oozie, Storm,Zookeeper.
> 2) Enable security.
> 3) Customize some properties: hadoop.proxyuser.yarn.hosts, hadoop.proxyuser.HTTP.groups, oozie.authentication.type, yarn.timeline-service.http-authentication.type.
> 4) Export blueprint config. Check that customized properties are correct in exported config.
> 5) Deploy kerberized cluster via blueprints.
> 6) Check that after deploy customized properties are the same as they were set during customization.
> Result: customized properties were reverted. Looks like during BP deploy, we are just updating all kerberos properties with values from kerberos.json from related service
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 739cd43 
>   ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java 9390b3a 
> 
> Diff: https://reviews.apache.org/r/42306/diff/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 42306: Kerberos specifical properties reverted during secured cluster deploy via blueprint

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42306/
-----------------------------------------------------------

(Updated Січ. 14, 2016, 6:54 після полудня)


Review request for Ambari, Robert Levas and Robert Nettleton.


Bugs: AMBARI-14668
    https://issues.apache.org/jira/browse/AMBARI-14668


Repository: ambari


Description
-------

1) Deploy cluster with HDFS,YARN,Oozie, Storm,Zookeeper.
2) Enable security.
3) Customize some properties: hadoop.proxyuser.yarn.hosts, hadoop.proxyuser.HTTP.groups, oozie.authentication.type, yarn.timeline-service.http-authentication.type.
4) Export blueprint config. Check that customized properties are correct in exported config.
5) Deploy kerberized cluster via blueprints.
6) Check that after deploy customized properties are the same as they were set during customization.
Result: customized properties were reverted. Looks like during BP deploy, we are just updating all kerberos properties with values from kerberos.json from related service


Diffs (updated)
-----

  ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 739cd43 
  ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java 9390b3a 

Diff: https://reviews.apache.org/r/42306/diff/


Testing
-------

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 42306: Kerberos specifical properties reverted during secured cluster deploy via blueprint

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.

(Updated Січ. 14, 2016, 6:48 після полудня)

Review request for Ambari, Robert Levas and Robert Nettleton.

Bugs: AMBARI-14668
https://issues.apache.org/jira/browse/AMBARI-14668

Repository: ambari

Description
-------

Diffs (updated)
-----

ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 739cd43
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java c885ca7
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java 9390b3a

Diff: https://reviews.apache.org/r/42306/diff/

Testing
-------

mvn clean test

Thanks,

Vitalyi Brodetskyi

Re: Review Request 42306: Kerberos specifical properties reverted during secured cluster deploy via blueprint

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.


> On Січ. 14, 2016, 4:02 після полудня, Robert Nettleton wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java, line 2274
> > <https://reviews.apache.org/r/42306/diff/2/?file=1196918#file1196918line2274>
> >
> >     I'm almost certain that the "INITIAL_INSTALL" phase is only set when a Blueprints deployment is used.  
> >     
> >     @smohanty can comment on this further. 
> >     
> >     Adding the new "deployType" of "BLUEPRINT" is fine, but may not be necessary, considering my comment above.

I have checked...we are really using INITIAL_INSTALL only for blueprint install


> On Січ. 14, 2016, 4:02 після полудня, Robert Nettleton wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java, line 369
> > <https://reviews.apache.org/r/42306/diff/2/?file=1196919#file1196919line369>
> >
> >     Again, I'm not sure this separate "deployType" is really needed, since the "INITIAL_INSTALL" phase is set. 
> >     
> >     That being said, it wouldn't necessarily be a bad idea to have this deployType set explicitly.  
> >     
> >     I'm ok with this either way, just wanted to register this comment for possible further review if necessary.

I have checked...we are really using INITIAL_INSTALL only for blueprint install


> On Січ. 14, 2016, 4:02 після полудня, Robert Nettleton wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java, line 125
> > <https://reviews.apache.org/r/42306/diff/2/?file=1196920#file1196920line125>
> >
> >     The Stack class can be used to determine if a property is a customized value, or is just a stack default.  
> >     
> >     Please see:
> >     
> >     org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor#doFilterStackDefaults
> >     
> >     for an example of how to use the Stack class to determine if the current config property is equal to it's stack default. 
> >     
> >     I'm not sure there's any way to determine the difference between the stack default and a user copying the stack default.

I don't think we should fix users issues. If user set property in blueprint, then we are just using it.


- Vitalyi


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42306/#review114488
-----------------------------------------------------------


On Січ. 14, 2016, 6:48 після полудня, Vitalyi Brodetskyi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42306/
> -----------------------------------------------------------
> 
> (Updated Січ. 14, 2016, 6:48 після полудня)
> 
> 
> Review request for Ambari, Robert Levas and Robert Nettleton.
> 
> 
> Bugs: AMBARI-14668
>     https://issues.apache.org/jira/browse/AMBARI-14668
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 1) Deploy cluster with HDFS,YARN,Oozie, Storm,Zookeeper.
> 2) Enable security.
> 3) Customize some properties: hadoop.proxyuser.yarn.hosts, hadoop.proxyuser.HTTP.groups, oozie.authentication.type, yarn.timeline-service.http-authentication.type.
> 4) Export blueprint config. Check that customized properties are correct in exported config.
> 5) Deploy kerberized cluster via blueprints.
> 6) Check that after deploy customized properties are the same as they were set during customization.
> Result: customized properties were reverted. Looks like during BP deploy, we are just updating all kerberos properties with values from kerberos.json from related service
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 739cd43 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java c885ca7 
>   ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java 9390b3a 
> 
> Diff: https://reviews.apache.org/r/42306/diff/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 42306: Kerberos specifical properties reverted during secured cluster deploy via blueprint

Posted by Robert Nettleton <rn...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42306/#review114488
-----------------------------------------------------------

Ship it!



ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java (line 2274)
<https://reviews.apache.org/r/42306/#comment175338>

    I'm almost certain that the "INITIAL_INSTALL" phase is only set when a Blueprints deployment is used.  
    
    @smohanty can comment on this further. 
    
    Adding the new "deployType" of "BLUEPRINT" is fine, but may not be necessary, considering my comment above.



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java (line 361)
<https://reviews.apache.org/r/42306/#comment175339>

    Again, I'm not sure this separate "deployType" is really needed, since the "INITIAL_INSTALL" phase is set. 
    
    That being said, it wouldn't necessarily be a bad idea to have this deployType set explicitly.  
    
    I'm ok with this either way, just wanted to register this comment for possible further review if necessary.



ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java (line 125)
<https://reviews.apache.org/r/42306/#comment175340>

    The Stack class can be used to determine if a property is a customized value, or is just a stack default.  
    
    Please see:
    
    org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor#doFilterStackDefaults
    
    for an example of how to use the Stack class to determine if the current config property is equal to it's stack default. 
    
    I'm not sure there's any way to determine the difference between the stack default and a user copying the stack default.


- Robert Nettleton


On Jan. 14, 2016, 2:59 p.m., Vitalyi Brodetskyi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42306/
> -----------------------------------------------------------
> 
> (Updated Jan. 14, 2016, 2:59 p.m.)
> 
> 
> Review request for Ambari, Robert Levas and Robert Nettleton.
> 
> 
> Bugs: AMBARI-14668
>     https://issues.apache.org/jira/browse/AMBARI-14668
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 1) Deploy cluster with HDFS,YARN,Oozie, Storm,Zookeeper.
> 2) Enable security.
> 3) Customize some properties: hadoop.proxyuser.yarn.hosts, hadoop.proxyuser.HTTP.groups, oozie.authentication.type, yarn.timeline-service.http-authentication.type.
> 4) Export blueprint config. Check that customized properties are correct in exported config.
> 5) Deploy kerberized cluster via blueprints.
> 6) Check that after deploy customized properties are the same as they were set during customization.
> Result: customized properties were reverted. Looks like during BP deploy, we are just updating all kerberos properties with values from kerberos.json from related service
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 739cd43 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java c885ca7 
>   ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java 9390b3a 
> 
> Diff: https://reviews.apache.org/r/42306/diff/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 42306: Kerberos specifical properties reverted during secured cluster deploy via blueprint

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.

(Updated Січ. 14, 2016, 2:59 після полудня)

Review request for Ambari, Robert Levas and Robert Nettleton.

Bugs: AMBARI-14668
https://issues.apache.org/jira/browse/AMBARI-14668

Repository: ambari

Description
-------

Diffs (updated)
-----

ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 739cd43
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java c885ca7
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java 9390b3a

Diff: https://reviews.apache.org/r/42306/diff/

Testing
-------

mvn clean test

Thanks,

Vitalyi Brodetskyi