[jira] [Assigned] (METRON-1177) Stale running topologies seen post-kerberization and cause exceptions

["Nick Allen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/METRON-1177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nick Allen reassigned METRON-1177:
----------------------------------

    Assignee: Nick Allen

> Stale running topologies seen post-kerberization and cause exceptions
> ---------------------------------------------------------------------
>
>                 Key: METRON-1177
>                 URL: https://issues.apache.org/jira/browse/METRON-1177
>             Project: Metron
>          Issue Type: Bug
>            Reporter: Anand Subramanian
>            Assignee: Nick Allen
>
> *Steps to Reproduce*
> 1. Install Ambari on 12 node cluster
> 2. Install Metron using MPack
> 3. Metron topologies are now running as user 'storm'
> 4. Setup standalone KDC on Ambari node by following these [steps|https://github.com/apache/metron/blob/master/metron-deployment/Kerberos-manual-setup.md#setup-a-kdc]
> 5. Initiate "kerberize" cluster using Ambari
> 6. Ambari triggers "Stop all services"
> 7. In the event of a race condition, the REST service shuts down (which prevents from being able to shutdown metron topologies)
> 8. Metron topology shut down occurs, but Metron Mpack cannot stop topologies because REST already stopped. Metron ignores this condition and continues on assuming the topologies are down
> 9. Storm service shuts down
> 10. Cluster kerberization occurs
> 11. Ambari triggers "Start all services"
> 12. Storm service starts. 
> 13. Since the Metron topologies were running when Storm was shutdown, these same topologies are attempted to be started again by Storm, but are declared as "already running"
> 14. All other services start successfully
> 15. Now, if we attempt to run a test with the stale topologies, following exceptions are seen:
> {code}
> AuthorizationException(msg:getTopologyInfo on topology snort is not authorized)
>         at org.apache.storm.generated.Nimbus$getTopologyInfo_result$getTopologyInfo_resultStandardScheme.read(Nimbus.java:36971)
>         at org.apache.storm.generated.Nimbus$getTopologyInfo_result$getTopologyInfo_resultStandardScheme.read(Nimbus.java:36939)
>         at org.apache.storm.generated.Nimbus$getTopologyInfo_result.read(Nimbus.java:36870)
>         at org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
>         at org.apache.storm.generated.Nimbus$Client.recv_getTopologyInfo(Nimbus.java:1182)
>         at org.apache.storm.generated.Nimbus$Client.getTopologyInfo(Nimbus.java:1169)
>         at org.hw.metron.lib.storm.StormClient.isTopologyRunning(StormClient.java:150)
>         at org.hw.metron.lib.storm.StormClient.startTopology(StormClient.java:68)
>         at functional.SnortTopologyTest.snortElasticsearchIndexTest(SnortTopologyTest.java:36)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:86)
>         at org.testng.internal.Invoker.invokeMethod(Invoker.java:643)
>         at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:820)
>         at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1128)
>         at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
>         at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
>         at org.testng.TestRunner.privateRun(TestRunner.java:782)
>         at org.testng.TestRunner.run(TestRunner.java:632)
>         at org.testng.SuiteRunner.runTest(SuiteRunner.java:366)
>         at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:361)
>         at org.testng.SuiteRunner.privateRun(SuiteRunner.java:319)
>         at org.testng.SuiteRunner.run(SuiteRunner.java:268)
>         at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
>         at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
>         at org.testng.TestNG.runSuitesSequentially(TestNG.java:1244)
>         at org.testng.TestNG.runSuitesLocally(TestNG.java:1169)
>         at org.testng.TestNG.run(TestNG.java:1064)
>         at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:295)
>         at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:84)
>         at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:90)
>         at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
>         at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
>         at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)