You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@ambari.apache.org by Sebastian Toader <st...@hortonworks.com> on 2016/03/02 13:12:10 UTC

Review Request 44206: NPE thrown when secure cluster is created with Blueprint that includes ACCUMULO

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44206/
-----------------------------------------------------------

Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Levas, and Robert Nettleton.

Bugs: AMBARI-15220
https://issues.apache.org/jira/browse/AMBARI-15220

Repository: ambari

Description
-------

Ambari server load service properties from stack service definition xml files. These properties than are stored in a map where the key is the config type of the property (the xml file that contains the property eg. core-site.xml). There might config types that have no properties defined at all like ACCUMULO's client.xml thus these one were missed to be loaded.
The side effect of this behavior is that when a cluster is configured when it's created using Blueprint and INIT config version was only created for the config types that had properties defined the others were not persisted into the database.

If any of the config types that had no INIT version was updated later than the server threw a NPE while waiting for the config type to get into TOPOLOGY_RESOLVED state.

In this particular case no INIT version of "client" config type was saved into the clusterconfig table for ACCUMULO service. Since this was a secure cluster being created using Blueprint the kerberos related actions were carried out which updated "client" config type by adding "kerberos.server.primary": "{{bare_accumulo_principal}}" to it. This led to a NPE in the application.

Stack initialisation has been modified such as after loading service properties from stack definition xml files it loads also the the empty config types as well.

Diffs
-----

ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java 7f911e9
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackTest.java 65f94aa

Diff: https://reviews.apache.org/r/44206/diff/

Testing
-------

Manual tests:
1. Provision secured cluster using a Blueprint that includes ACCUMULO.
2. Provisioned a cluster with ACCUMULO included using the Web UI than enabled kerberos on the cluster.

Unit tests currently running.

Thanks,

Sebastian Toader

Re: Review Request 44206: NPE thrown when secure cluster is created with Blueprint that includes ACCUMULO

Posted by Oliver Szabo <os...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44206/#review121632
-----------------------------------------------------------




ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java (line 692)
<https://reviews.apache.org/r/44206/#comment183380>

    Just a typo, I guess it should be 'with' instead of 'we'.


- Oliver Szabo


On March 2, 2016, 12:12 p.m., Sebastian Toader wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44206/
> -----------------------------------------------------------
> 
> (Updated March 2, 2016, 12:12 p.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Levas, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-15220
>     https://issues.apache.org/jira/browse/AMBARI-15220
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Ambari server load service properties from stack service definition xml files. These properties than are stored in a map where the key is the config type of the property (the xml file that contains the property eg. core-site.xml). There might config types that have no properties defined at all like ACCUMULO's client.xml thus these one were missed to be loaded. 
> The side effect of this behavior is that when a cluster is configured when it's created using Blueprint and INIT config version was only created for the config types that had properties defined the others were not persisted into the database. 
> 
> If any of the config types that had no INIT version was updated later than the server threw a NPE while waiting for the config type to get into TOPOLOGY_RESOLVED state.
> 
> In this particular case no INIT version of "client" config type was saved into the clusterconfig table for ACCUMULO service. Since this was a secure cluster being created using Blueprint the kerberos related actions were carried out which updated "client" config type by adding "kerberos.server.primary": "{{bare_accumulo_principal}}" to it. This led to a NPE in the application.
> 
> 
> Stack initialisation has been modified such as after loading service properties from stack definition xml files it loads also the the empty config types as well.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java 7f911e9 
>   ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackTest.java 65f94aa 
> 
> Diff: https://reviews.apache.org/r/44206/diff/
> 
> 
> Testing
> -------
> 
> Manual tests:
> 1. Provision secured cluster using a Blueprint that includes ACCUMULO.
> 2. Provisioned a cluster with ACCUMULO included using the Web UI than enabled kerberos on the cluster.
> 
> Unit tests currently running.
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>

Re: Review Request 44206: NPE thrown when secure cluster is created with Blueprint that includes ACCUMULO

Posted by Laszlo Puskas <lp...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44206/#review121631
-----------------------------------------------------------


Ship it!




Ship It!

- Laszlo Puskas


On March 2, 2016, 12:12 p.m., Sebastian Toader wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44206/
> -----------------------------------------------------------
> 
> (Updated March 2, 2016, 12:12 p.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Levas, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-15220
>     https://issues.apache.org/jira/browse/AMBARI-15220
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Ambari server load service properties from stack service definition xml files. These properties than are stored in a map where the key is the config type of the property (the xml file that contains the property eg. core-site.xml). There might config types that have no properties defined at all like ACCUMULO's client.xml thus these one were missed to be loaded. 
> The side effect of this behavior is that when a cluster is configured when it's created using Blueprint and INIT config version was only created for the config types that had properties defined the others were not persisted into the database. 
> 
> If any of the config types that had no INIT version was updated later than the server threw a NPE while waiting for the config type to get into TOPOLOGY_RESOLVED state.
> 
> In this particular case no INIT version of "client" config type was saved into the clusterconfig table for ACCUMULO service. Since this was a secure cluster being created using Blueprint the kerberos related actions were carried out which updated "client" config type by adding "kerberos.server.primary": "{{bare_accumulo_principal}}" to it. This led to a NPE in the application.
> 
> 
> Stack initialisation has been modified such as after loading service properties from stack definition xml files it loads also the the empty config types as well.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java 7f911e9 
>   ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackTest.java 65f94aa 
> 
> Diff: https://reviews.apache.org/r/44206/diff/
> 
> 
> Testing
> -------
> 
> Manual tests:
> 1. Provision secured cluster using a Blueprint that includes ACCUMULO.
> 2. Provisioned a cluster with ACCUMULO included using the Web UI than enabled kerberos on the cluster.
> 
> Unit tests currently running.
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>

Re: Review Request 44206: NPE thrown when secure cluster is created with Blueprint that includes ACCUMULO

Posted by Oliver Szabo <os...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44206/#review121636
-----------------------------------------------------------


Ship it!




Ship It!

- Oliver Szabo


On March 2, 2016, 1:53 p.m., Sebastian Toader wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44206/
> -----------------------------------------------------------
> 
> (Updated March 2, 2016, 1:53 p.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Levas, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-15220
>     https://issues.apache.org/jira/browse/AMBARI-15220
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Ambari server load service properties from stack service definition xml files. These properties than are stored in a map where the key is the config type of the property (the xml file that contains the property eg. core-site.xml). There might config types that have no properties defined at all like ACCUMULO's client.xml thus these one were missed to be loaded. 
> The side effect of this behavior is that when a cluster is configured when it's created using Blueprint and INIT config version was only created for the config types that had properties defined the others were not persisted into the database. 
> 
> If any of the config types that had no INIT version was updated later than the server threw a NPE while waiting for the config type to get into TOPOLOGY_RESOLVED state.
> 
> In this particular case no INIT version of "client" config type was saved into the clusterconfig table for ACCUMULO service. Since this was a secure cluster being created using Blueprint the kerberos related actions were carried out which updated "client" config type by adding "kerberos.server.primary": "{{bare_accumulo_principal}}" to it. This led to a NPE in the application.
> 
> 
> Stack initialisation has been modified such as after loading service properties from stack definition xml files it loads also the the empty config types as well.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java 7f911e9 
>   ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackTest.java 65f94aa 
> 
> Diff: https://reviews.apache.org/r/44206/diff/
> 
> 
> Testing
> -------
> 
> Manual tests:
> 1. Provision secured cluster using a Blueprint that includes ACCUMULO.
> 2. Provisioned a cluster with ACCUMULO included using the Web UI than enabled kerberos on the cluster.
> 
> Ambari-server unit tests results:
> Results :
> 
> Tests run: 3897, Failures: 0, Errors: 0, Skipped: 33
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>

Re: Review Request 44206: NPE thrown when secure cluster is created with Blueprint that includes ACCUMULO

Posted by Robert Nettleton <rn...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44206/#review121664
-----------------------------------------------------------


Ship it!




Ship It!

- Robert Nettleton


On March 2, 2016, 1:53 p.m., Sebastian Toader wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44206/
> -----------------------------------------------------------
> 
> (Updated March 2, 2016, 1:53 p.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Levas, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-15220
>     https://issues.apache.org/jira/browse/AMBARI-15220
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Ambari server load service properties from stack service definition xml files. These properties than are stored in a map where the key is the config type of the property (the xml file that contains the property eg. core-site.xml). There might config types that have no properties defined at all like ACCUMULO's client.xml thus these one were missed to be loaded. 
> The side effect of this behavior is that when a cluster is configured when it's created using Blueprint and INIT config version was only created for the config types that had properties defined the others were not persisted into the database. 
> 
> If any of the config types that had no INIT version was updated later than the server threw a NPE while waiting for the config type to get into TOPOLOGY_RESOLVED state.
> 
> In this particular case no INIT version of "client" config type was saved into the clusterconfig table for ACCUMULO service. Since this was a secure cluster being created using Blueprint the kerberos related actions were carried out which updated "client" config type by adding "kerberos.server.primary": "{{bare_accumulo_principal}}" to it. This led to a NPE in the application.
> 
> 
> Stack initialisation has been modified such as after loading service properties from stack definition xml files it loads also the the empty config types as well.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java 7f911e9 
>   ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackTest.java 65f94aa 
> 
> Diff: https://reviews.apache.org/r/44206/diff/
> 
> 
> Testing
> -------
> 
> Manual tests:
> 1. Provision secured cluster using a Blueprint that includes ACCUMULO.
> 2. Provisioned a cluster with ACCUMULO included using the Web UI than enabled kerberos on the cluster.
> 
> Ambari-server unit tests results:
> Results :
> 
> Tests run: 3897, Failures: 0, Errors: 0, Skipped: 33
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>

Re: Review Request 44206: NPE thrown when secure cluster is created with Blueprint that includes ACCUMULO

Posted by Robert Levas <rl...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44206/#review121648
-----------------------------------------------------------


Ship it!




Ship It!

- Robert Levas


On March 2, 2016, 8:53 a.m., Sebastian Toader wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44206/
> -----------------------------------------------------------
> 
> (Updated March 2, 2016, 8:53 a.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Levas, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-15220
>     https://issues.apache.org/jira/browse/AMBARI-15220
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Ambari server load service properties from stack service definition xml files. These properties than are stored in a map where the key is the config type of the property (the xml file that contains the property eg. core-site.xml). There might config types that have no properties defined at all like ACCUMULO's client.xml thus these one were missed to be loaded. 
> The side effect of this behavior is that when a cluster is configured when it's created using Blueprint and INIT config version was only created for the config types that had properties defined the others were not persisted into the database. 
> 
> If any of the config types that had no INIT version was updated later than the server threw a NPE while waiting for the config type to get into TOPOLOGY_RESOLVED state.
> 
> In this particular case no INIT version of "client" config type was saved into the clusterconfig table for ACCUMULO service. Since this was a secure cluster being created using Blueprint the kerberos related actions were carried out which updated "client" config type by adding "kerberos.server.primary": "{{bare_accumulo_principal}}" to it. This led to a NPE in the application.
> 
> 
> Stack initialisation has been modified such as after loading service properties from stack definition xml files it loads also the the empty config types as well.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java 7f911e9 
>   ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackTest.java 65f94aa 
> 
> Diff: https://reviews.apache.org/r/44206/diff/
> 
> 
> Testing
> -------
> 
> Manual tests:
> 1. Provision secured cluster using a Blueprint that includes ACCUMULO.
> 2. Provisioned a cluster with ACCUMULO included using the Web UI than enabled kerberos on the cluster.
> 
> Ambari-server unit tests results:
> Results :
> 
> Tests run: 3897, Failures: 0, Errors: 0, Skipped: 33
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>

Re: Review Request 44206: NPE thrown when secure cluster is created with Blueprint that includes ACCUMULO

Posted by Sebastian Toader <st...@hortonworks.com>.

(Updated March 2, 2016, 2:53 p.m.)

Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Levas, and Robert Nettleton.

Changes
-------

Rebase with latest trunk

Bugs: AMBARI-15220
https://issues.apache.org/jira/browse/AMBARI-15220

Repository: ambari

Description
-------

If any of the config types that had no INIT version was updated later than the server threw a NPE while waiting for the config type to get into TOPOLOGY_RESOLVED state.

Stack initialisation has been modified such as after loading service properties from stack definition xml files it loads also the the empty config types as well.

Diffs (updated)
-----

ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java 7f911e9
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackTest.java 65f94aa

Diff: https://reviews.apache.org/r/44206/diff/

Testing (updated)
-------

Manual tests:
1. Provision secured cluster using a Blueprint that includes ACCUMULO.
2. Provisioned a cluster with ACCUMULO included using the Web UI than enabled kerberos on the cluster.

Ambari-server unit tests results:
Results :

Tests run: 3897, Failures: 0, Errors: 0, Skipped: 33

Thanks,

Sebastian Toader