You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Alejandro Fernandez <af...@hortonworks.com> on 2014/11/04 01:20:41 UTC
Review Request 27549: Storm in secure Ambari cluster has config null
error when deploying a topology
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27549/
-----------------------------------------------------------
Review request for Ambari, Mahadev Konar, Sumit Mohanty, and Sid Wagle.
Bugs: AMBARI-8131
https://issues.apache.org/jira/browse/AMBARI-8131
Repository: ambari
Description
-------
Create a secure cluster with Storm, kinit as ambari-qa, then attempt to run the following command to add a topology, which fails,
```
[root@alejandrohdp22-1 tmp]# /usr/hdp/current/storm-client/bin/storm -c java.security.auth.login.config=/etc/storm/conf/client_jaas.conf jar /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1
Running: java -client -Dstorm.options=java.security.auth.login.config=/etc/storm/conf/client_jaas.conf -Dstorm.home=/usr/hdp/2.2.0.0-1744/storm -Dstorm.log.dir=/usr/hdp/2.2.0.0-1744/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib -Dstorm.conf.file= -cp /usr/hdp/2.2.0.0-1744/storm/lib/asm-4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-security-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gson-2.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/zookeeper.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-stacktrace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/snakeyaml-1.11.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-random-1.2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-cred-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-lang-2.5.jar:/usr/hdp/2.2.0.0-1744/s
torm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.2.0.0-1744/storm/lib/mysql-connector-java.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.cli-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/chill-java-0.3.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-http-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/eclipselink-2.5.2-M1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlets-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clojure-1.5.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-continuation-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-impl-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-util-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-fileupload-1.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-exec-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/kryo-2.21.jar:/usr/hdp/2.2.0.0-1744/storm/lib/compojure-1.1.3.jar:/usr/hdp/
2.2.0.0-1744/storm/lib/javax.persistence-2.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-configuration-1.10.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-common-2.6.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-core-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/storm-core-0.9.3.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/disruptor-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clout-1.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/json-simple-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.macro-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/slf4j-api-1.6.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-equality-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-anti-forgery-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlet-7.6.13.v20
130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/objenesis-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/minlog-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-time-0.4.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-client-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-server-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-collections-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/guava-11.0.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jersey-bundle-1.17.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/math.numeric-tower-0.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-classic-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-auth-2.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/comm
ons-io-2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-audit-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/carbonite-1.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-io-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jgrapht-core-0.9.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-storm-plugin-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/joda-time-2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-logging-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-common-0.4.0.2.2.0.0-1744.jar:/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar:/usr/hdp/2.2.0.0-1744/storm/conf:/usr/hdp/2.2.0.0-1744/storm/bin -Dstorm.jar=/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1
773 [main] INFO backtype.storm.StormSubmitter - Generated ZooKeeper secret payload for MD5-digest: -5766984969082465146:-6900977625499590902
788 [main] INFO backtype.storm.security.auth.AuthUtils - Got AutoCreds []
Exception in thread "main" java.lang.RuntimeException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: JavaLoginConfig, provider: SUN, class: sun.security.provider.ConfigSpiFile)
at backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:66)
at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:82)
at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:66)
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:52)
at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:36)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:211)
at backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:273)
at backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:254)
at storm.starter.ExclamationTopology.main(ExclamationTopology.java:76)
Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: JavaLoginConfig, provider: SUN, class: sun.security.provider.ConfigSpiFile)
at java.security.Provider$Service.newInstance(Provider.java:1259)
at sun.security.jca.GetInstance.getInstance(GetInstance.java:243)
at sun.security.jca.GetInstance.getInstance(GetInstance.java:190)
at javax.security.auth.login.Configuration.getInstance(Configuration.java:352)
at backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:64)
... 8 more
Caused by: java.io.IOException: Configuration Error:
Line 1: expected [{], found [null]
at com.sun.security.auth.login.ConfigFile.match(ConfigFile.java:572)
at com.sun.security.auth.login.ConfigFile.parseLoginEntry(ConfigFile.java:405)
at com.sun.security.auth.login.ConfigFile.readConfig(ConfigFile.java:383)
at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:283)
at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:166)
at com.sun.security.auth.login.ConfigFile.<init>(ConfigFile.java:124)
at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:72)
at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:61)
#
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.provider.ConfigSpiFile.<init>(ConfigSpiFile.java:61)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at java.security.Provider$Service.newInstance(Provider.java:1254)
... 12 more
```
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py 32cb60d
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py 5699e57
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2 cf78af4
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2 82ff239
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2 7c650df
Diff: https://reviews.apache.org/r/27549/diff/
Testing
-------
Ran unit tests on ambari-server,
----------------------------------------------------------------------
Total run:684
Total errors:0
Total failures:0
OK
Reproduced the issue on a live cluster, and was able to get the topology added.
E.g.,
su - ambari-qa
kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa@EXAMPLE.COM
scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py root@162.216.149.188:/tmp
scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py root@162.216.149.188:/tmp
scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2 root@162.216.149.188:/tmp
scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2 root@162.216.149.188:/tmp
scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2 root@162.216.149.188:/tmp
scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2 root@162.216.149.188:/tmp
yes | cp /tmp/params.py /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/params.py
yes | cp /tmp/storm.py /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/storm.py
yes | cp /tmp/client_jaas.conf.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2
yes | cp /tmp/storm.yaml.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2
yes | cp /tmp/storm_jaas.conf.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2
yes | cp /tmp/worker-launcher.cfg.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2
ambari-server restart (to reload templates)
Ensure that client_jaas.conf is in /etc/storm/conf/ . Then run,
/usr/hdp/current/storm-client/bin/storm -c java.security.auth.login.config=/etc/storm/conf/client_jaas.conf jar /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1
Thanks,
Alejandro Fernandez
Re: Review Request 27549: Storm in secure Ambari cluster has config
null error when deploying a topology
Posted by Mahadev Konar <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27549/#review59686
-----------------------------------------------------------
Ship it!
Ship It!
- Mahadev Konar
On Nov. 4, 2014, 12:20 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27549/
> -----------------------------------------------------------
>
> (Updated Nov. 4, 2014, 12:20 a.m.)
>
>
> Review request for Ambari, Mahadev Konar, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-8131
> https://issues.apache.org/jira/browse/AMBARI-8131
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Create a secure cluster with Storm, kinit as ambari-qa, then attempt to run the following command to add a topology, which fails,
>
> ```
> [root@alejandrohdp22-1 tmp]# /usr/hdp/current/storm-client/bin/storm -c java.security.auth.login.config=/etc/storm/conf/client_jaas.conf jar /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1
> Running: java -client -Dstorm.options=java.security.auth.login.config=/etc/storm/conf/client_jaas.conf -Dstorm.home=/usr/hdp/2.2.0.0-1744/storm -Dstorm.log.dir=/usr/hdp/2.2.0.0-1744/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib -Dstorm.conf.file= -cp /usr/hdp/2.2.0.0-1744/storm/lib/asm-4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-security-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gson-2.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/zookeeper.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-stacktrace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/snakeyaml-1.11.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-random-1.2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-cred-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-lang-2.5.jar:/usr/hdp/2.2.0.0-1744
/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.2.0.0-1744/storm/lib/mysql-connector-java.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.cli-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/chill-java-0.3.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-http-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/eclipselink-2.5.2-M1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlets-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clojure-1.5.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-continuation-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-impl-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-util-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-fileupload-1.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-exec-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/kryo-2.21.jar:/usr/hdp/2.2.0.0-1744/storm/lib/compojure-1.1.3.jar:/usr/hd
p/2.2.0.0-1744/storm/lib/javax.persistence-2.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-configuration-1.10.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-common-2.6.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-core-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/storm-core-0.9.3.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/disruptor-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clout-1.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/json-simple-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.macro-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/slf4j-api-1.6.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-equality-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-anti-forgery-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlet-7.6.13.v
20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/objenesis-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/minlog-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-time-0.4.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-client-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-server-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-collections-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/guava-11.0.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jersey-bundle-1.17.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/math.numeric-tower-0.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-classic-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-auth-2.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/co
mmons-io-2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-audit-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/carbonite-1.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-io-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jgrapht-core-0.9.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-storm-plugin-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/joda-time-2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-logging-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-common-0.4.0.2.2.0.0-1744.jar:/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar:/usr/hdp/2.2.0.0-1744/storm/conf:/usr/hdp/2.2.0.0-1744/storm/bin -Dstorm.jar=/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1
> 773 [main] INFO backtype.storm.StormSubmitter - Generated ZooKeeper secret payload for MD5-digest: -5766984969082465146:-6900977625499590902
> 788 [main] INFO backtype.storm.security.auth.AuthUtils - Got AutoCreds []
> Exception in thread "main" java.lang.RuntimeException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: JavaLoginConfig, provider: SUN, class: sun.security.provider.ConfigSpiFile)
> at backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:66)
> at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:82)
> at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:66)
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:52)
> at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:36)
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:211)
> at backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:273)
> at backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:254)
> at storm.starter.ExclamationTopology.main(ExclamationTopology.java:76)
> Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: JavaLoginConfig, provider: SUN, class: sun.security.provider.ConfigSpiFile)
> at java.security.Provider$Service.newInstance(Provider.java:1259)
> at sun.security.jca.GetInstance.getInstance(GetInstance.java:243)
> at sun.security.jca.GetInstance.getInstance(GetInstance.java:190)
> at javax.security.auth.login.Configuration.getInstance(Configuration.java:352)
> at backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:64)
> ... 8 more
> Caused by: java.io.IOException: Configuration Error:
> Line 1: expected [{], found [null]
> at com.sun.security.auth.login.ConfigFile.match(ConfigFile.java:572)
> at com.sun.security.auth.login.ConfigFile.parseLoginEntry(ConfigFile.java:405)
> at com.sun.security.auth.login.ConfigFile.readConfig(ConfigFile.java:383)
> at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:283)
> at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:166)
> at com.sun.security.auth.login.ConfigFile.<init>(ConfigFile.java:124)
> at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:72)
> at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:61)
> #
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.security.provider.ConfigSpiFile.<init>(ConfigSpiFile.java:61)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at java.security.Provider$Service.newInstance(Provider.java:1254)
> ... 12 more
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py 32cb60d
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py 5699e57
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2 cf78af4
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2 82ff239
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2 7c650df
>
> Diff: https://reviews.apache.org/r/27549/diff/
>
>
> Testing
> -------
>
> Ran unit tests on ambari-server,
> ----------------------------------------------------------------------
> Total run:684
> Total errors:0
> Total failures:0
> OK
>
>
> Reproduced the issue on a live cluster, and was able to get the topology added.
> E.g.,
> su - ambari-qa
> kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa@EXAMPLE.COM
>
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2 root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2 root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2 root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2 root@162.216.149.188:/tmp
>
>
> yes | cp /tmp/params.py /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/params.py
> yes | cp /tmp/storm.py /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/storm.py
> yes | cp /tmp/client_jaas.conf.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2
> yes | cp /tmp/storm.yaml.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2
> yes | cp /tmp/storm_jaas.conf.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2
> yes | cp /tmp/worker-launcher.cfg.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2
>
> ambari-server restart (to reload templates)
>
> Ensure that client_jaas.conf is in /etc/storm/conf/ . Then run,
> /usr/hdp/current/storm-client/bin/storm -c java.security.auth.login.config=/etc/storm/conf/client_jaas.conf jar /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 27549: Storm in secure Ambari cluster has config
null error when deploying a topology
Posted by Sid Wagle <sw...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27549/#review59689
-----------------------------------------------------------
Ship it!
Ship It!
- Sid Wagle
On Nov. 4, 2014, 12:20 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27549/
> -----------------------------------------------------------
>
> (Updated Nov. 4, 2014, 12:20 a.m.)
>
>
> Review request for Ambari, Mahadev Konar, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-8131
> https://issues.apache.org/jira/browse/AMBARI-8131
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Create a secure cluster with Storm, kinit as ambari-qa, then attempt to run the following command to add a topology, which fails,
>
> ```
> [root@alejandrohdp22-1 tmp]# /usr/hdp/current/storm-client/bin/storm -c java.security.auth.login.config=/etc/storm/conf/client_jaas.conf jar /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1
> Running: java -client -Dstorm.options=java.security.auth.login.config=/etc/storm/conf/client_jaas.conf -Dstorm.home=/usr/hdp/2.2.0.0-1744/storm -Dstorm.log.dir=/usr/hdp/2.2.0.0-1744/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib -Dstorm.conf.file= -cp /usr/hdp/2.2.0.0-1744/storm/lib/asm-4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-security-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gson-2.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/zookeeper.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-stacktrace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/snakeyaml-1.11.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-random-1.2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-cred-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-lang-2.5.jar:/usr/hdp/2.2.0.0-1744
/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.2.0.0-1744/storm/lib/mysql-connector-java.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.cli-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/chill-java-0.3.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-http-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/eclipselink-2.5.2-M1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlets-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clojure-1.5.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-continuation-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-impl-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-util-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-fileupload-1.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-exec-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/kryo-2.21.jar:/usr/hdp/2.2.0.0-1744/storm/lib/compojure-1.1.3.jar:/usr/hd
p/2.2.0.0-1744/storm/lib/javax.persistence-2.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-configuration-1.10.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-common-2.6.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-core-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/storm-core-0.9.3.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/disruptor-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clout-1.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/json-simple-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.macro-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/slf4j-api-1.6.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-equality-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-anti-forgery-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlet-7.6.13.v
20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/objenesis-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/minlog-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-time-0.4.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-client-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-server-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-collections-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/guava-11.0.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jersey-bundle-1.17.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/math.numeric-tower-0.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-classic-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-auth-2.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/co
mmons-io-2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-audit-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/carbonite-1.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-io-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jgrapht-core-0.9.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-storm-plugin-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/joda-time-2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-logging-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-common-0.4.0.2.2.0.0-1744.jar:/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar:/usr/hdp/2.2.0.0-1744/storm/conf:/usr/hdp/2.2.0.0-1744/storm/bin -Dstorm.jar=/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1
> 773 [main] INFO backtype.storm.StormSubmitter - Generated ZooKeeper secret payload for MD5-digest: -5766984969082465146:-6900977625499590902
> 788 [main] INFO backtype.storm.security.auth.AuthUtils - Got AutoCreds []
> Exception in thread "main" java.lang.RuntimeException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: JavaLoginConfig, provider: SUN, class: sun.security.provider.ConfigSpiFile)
> at backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:66)
> at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:82)
> at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:66)
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:52)
> at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:36)
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:211)
> at backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:273)
> at backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:254)
> at storm.starter.ExclamationTopology.main(ExclamationTopology.java:76)
> Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: JavaLoginConfig, provider: SUN, class: sun.security.provider.ConfigSpiFile)
> at java.security.Provider$Service.newInstance(Provider.java:1259)
> at sun.security.jca.GetInstance.getInstance(GetInstance.java:243)
> at sun.security.jca.GetInstance.getInstance(GetInstance.java:190)
> at javax.security.auth.login.Configuration.getInstance(Configuration.java:352)
> at backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:64)
> ... 8 more
> Caused by: java.io.IOException: Configuration Error:
> Line 1: expected [{], found [null]
> at com.sun.security.auth.login.ConfigFile.match(ConfigFile.java:572)
> at com.sun.security.auth.login.ConfigFile.parseLoginEntry(ConfigFile.java:405)
> at com.sun.security.auth.login.ConfigFile.readConfig(ConfigFile.java:383)
> at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:283)
> at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:166)
> at com.sun.security.auth.login.ConfigFile.<init>(ConfigFile.java:124)
> at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:72)
> at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:61)
> #
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.security.provider.ConfigSpiFile.<init>(ConfigSpiFile.java:61)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at java.security.Provider$Service.newInstance(Provider.java:1254)
> ... 12 more
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py 32cb60d
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py 5699e57
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2 cf78af4
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2 82ff239
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2 7c650df
>
> Diff: https://reviews.apache.org/r/27549/diff/
>
>
> Testing
> -------
>
> Ran unit tests on ambari-server,
> ----------------------------------------------------------------------
> Total run:684
> Total errors:0
> Total failures:0
> OK
>
>
> Reproduced the issue on a live cluster, and was able to get the topology added.
> E.g.,
> su - ambari-qa
> kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa@EXAMPLE.COM
>
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2 root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2 root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2 root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2 root@162.216.149.188:/tmp
>
>
> yes | cp /tmp/params.py /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/params.py
> yes | cp /tmp/storm.py /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/storm.py
> yes | cp /tmp/client_jaas.conf.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2
> yes | cp /tmp/storm.yaml.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2
> yes | cp /tmp/storm_jaas.conf.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2
> yes | cp /tmp/worker-launcher.cfg.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2
>
> ambari-server restart (to reload templates)
>
> Ensure that client_jaas.conf is in /etc/storm/conf/ . Then run,
> /usr/hdp/current/storm-client/bin/storm -c java.security.auth.login.config=/etc/storm/conf/client_jaas.conf jar /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 27549: Storm in secure Ambari cluster has config
null error when deploying a topology
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27549/#review59685
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2
<https://reviews.apache.org/r/27549/#comment100985>
This is essentially the error that was preventing the command from passing.
- Alejandro Fernandez
On Nov. 4, 2014, 12:20 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27549/
> -----------------------------------------------------------
>
> (Updated Nov. 4, 2014, 12:20 a.m.)
>
>
> Review request for Ambari, Mahadev Konar, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-8131
> https://issues.apache.org/jira/browse/AMBARI-8131
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Create a secure cluster with Storm, kinit as ambari-qa, then attempt to run the following command to add a topology, which fails,
>
> ```
> [root@alejandrohdp22-1 tmp]# /usr/hdp/current/storm-client/bin/storm -c java.security.auth.login.config=/etc/storm/conf/client_jaas.conf jar /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1
> Running: java -client -Dstorm.options=java.security.auth.login.config=/etc/storm/conf/client_jaas.conf -Dstorm.home=/usr/hdp/2.2.0.0-1744/storm -Dstorm.log.dir=/usr/hdp/2.2.0.0-1744/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib -Dstorm.conf.file= -cp /usr/hdp/2.2.0.0-1744/storm/lib/asm-4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-security-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gson-2.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/zookeeper.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-stacktrace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/snakeyaml-1.11.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-random-1.2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-cred-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-lang-2.5.jar:/usr/hdp/2.2.0.0-1744
/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.2.0.0-1744/storm/lib/mysql-connector-java.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.cli-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/chill-java-0.3.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-http-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/eclipselink-2.5.2-M1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlets-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clojure-1.5.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-continuation-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-impl-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-util-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-fileupload-1.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-exec-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/kryo-2.21.jar:/usr/hdp/2.2.0.0-1744/storm/lib/compojure-1.1.3.jar:/usr/hd
p/2.2.0.0-1744/storm/lib/javax.persistence-2.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-configuration-1.10.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-common-2.6.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-core-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/storm-core-0.9.3.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/disruptor-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clout-1.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/json-simple-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.macro-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/slf4j-api-1.6.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-equality-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-anti-forgery-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlet-7.6.13.v
20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/objenesis-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/minlog-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-time-0.4.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-client-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-server-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-collections-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/guava-11.0.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jersey-bundle-1.17.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/math.numeric-tower-0.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-classic-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-auth-2.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/co
mmons-io-2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-audit-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/carbonite-1.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-io-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jgrapht-core-0.9.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-storm-plugin-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/joda-time-2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-logging-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-common-0.4.0.2.2.0.0-1744.jar:/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar:/usr/hdp/2.2.0.0-1744/storm/conf:/usr/hdp/2.2.0.0-1744/storm/bin -Dstorm.jar=/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1
> 773 [main] INFO backtype.storm.StormSubmitter - Generated ZooKeeper secret payload for MD5-digest: -5766984969082465146:-6900977625499590902
> 788 [main] INFO backtype.storm.security.auth.AuthUtils - Got AutoCreds []
> Exception in thread "main" java.lang.RuntimeException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: JavaLoginConfig, provider: SUN, class: sun.security.provider.ConfigSpiFile)
> at backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:66)
> at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:82)
> at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:66)
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:52)
> at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:36)
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:211)
> at backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:273)
> at backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:254)
> at storm.starter.ExclamationTopology.main(ExclamationTopology.java:76)
> Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: JavaLoginConfig, provider: SUN, class: sun.security.provider.ConfigSpiFile)
> at java.security.Provider$Service.newInstance(Provider.java:1259)
> at sun.security.jca.GetInstance.getInstance(GetInstance.java:243)
> at sun.security.jca.GetInstance.getInstance(GetInstance.java:190)
> at javax.security.auth.login.Configuration.getInstance(Configuration.java:352)
> at backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:64)
> ... 8 more
> Caused by: java.io.IOException: Configuration Error:
> Line 1: expected [{], found [null]
> at com.sun.security.auth.login.ConfigFile.match(ConfigFile.java:572)
> at com.sun.security.auth.login.ConfigFile.parseLoginEntry(ConfigFile.java:405)
> at com.sun.security.auth.login.ConfigFile.readConfig(ConfigFile.java:383)
> at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:283)
> at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:166)
> at com.sun.security.auth.login.ConfigFile.<init>(ConfigFile.java:124)
> at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:72)
> at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:61)
> #
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.security.provider.ConfigSpiFile.<init>(ConfigSpiFile.java:61)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at java.security.Provider$Service.newInstance(Provider.java:1254)
> ... 12 more
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py 32cb60d
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py 5699e57
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2 cf78af4
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2 82ff239
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2 7c650df
>
> Diff: https://reviews.apache.org/r/27549/diff/
>
>
> Testing
> -------
>
> Ran unit tests on ambari-server,
> ----------------------------------------------------------------------
> Total run:684
> Total errors:0
> Total failures:0
> OK
>
>
> Reproduced the issue on a live cluster, and was able to get the topology added.
> E.g.,
> su - ambari-qa
> kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa@EXAMPLE.COM
>
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2 root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2 root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2 root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2 root@162.216.149.188:/tmp
>
>
> yes | cp /tmp/params.py /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/params.py
> yes | cp /tmp/storm.py /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/storm.py
> yes | cp /tmp/client_jaas.conf.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2
> yes | cp /tmp/storm.yaml.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2
> yes | cp /tmp/storm_jaas.conf.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2
> yes | cp /tmp/worker-launcher.cfg.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2
>
> ambari-server restart (to reload templates)
>
> Ensure that client_jaas.conf is in /etc/storm/conf/ . Then run,
> /usr/hdp/current/storm-client/bin/storm -c java.security.auth.login.config=/etc/storm/conf/client_jaas.conf jar /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 27549: Storm in secure Ambari cluster has config
null error when deploying a topology
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27549/#review59688
-----------------------------------------------------------
Ship it!
Ship It!
- Sumit Mohanty
On Nov. 4, 2014, 12:20 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27549/
> -----------------------------------------------------------
>
> (Updated Nov. 4, 2014, 12:20 a.m.)
>
>
> Review request for Ambari, Mahadev Konar, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-8131
> https://issues.apache.org/jira/browse/AMBARI-8131
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Create a secure cluster with Storm, kinit as ambari-qa, then attempt to run the following command to add a topology, which fails,
>
> ```
> [root@alejandrohdp22-1 tmp]# /usr/hdp/current/storm-client/bin/storm -c java.security.auth.login.config=/etc/storm/conf/client_jaas.conf jar /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1
> Running: java -client -Dstorm.options=java.security.auth.login.config=/etc/storm/conf/client_jaas.conf -Dstorm.home=/usr/hdp/2.2.0.0-1744/storm -Dstorm.log.dir=/usr/hdp/2.2.0.0-1744/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib -Dstorm.conf.file= -cp /usr/hdp/2.2.0.0-1744/storm/lib/asm-4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-security-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gson-2.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/zookeeper.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-stacktrace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/snakeyaml-1.11.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-random-1.2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-cred-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-lang-2.5.jar:/usr/hdp/2.2.0.0-1744
/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.2.0.0-1744/storm/lib/mysql-connector-java.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.cli-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/chill-java-0.3.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-http-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/eclipselink-2.5.2-M1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlets-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clojure-1.5.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-continuation-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-impl-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-util-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-fileupload-1.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-exec-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/kryo-2.21.jar:/usr/hdp/2.2.0.0-1744/storm/lib/compojure-1.1.3.jar:/usr/hd
p/2.2.0.0-1744/storm/lib/javax.persistence-2.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-configuration-1.10.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-common-2.6.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-core-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/storm-core-0.9.3.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/disruptor-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clout-1.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/json-simple-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.macro-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/slf4j-api-1.6.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-equality-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-anti-forgery-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlet-7.6.13.v
20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/objenesis-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/minlog-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-time-0.4.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-client-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-server-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-collections-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/guava-11.0.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jersey-bundle-1.17.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/math.numeric-tower-0.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-classic-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-auth-2.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/co
mmons-io-2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-audit-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/carbonite-1.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-io-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jgrapht-core-0.9.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-storm-plugin-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/joda-time-2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-logging-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-common-0.4.0.2.2.0.0-1744.jar:/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar:/usr/hdp/2.2.0.0-1744/storm/conf:/usr/hdp/2.2.0.0-1744/storm/bin -Dstorm.jar=/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1
> 773 [main] INFO backtype.storm.StormSubmitter - Generated ZooKeeper secret payload for MD5-digest: -5766984969082465146:-6900977625499590902
> 788 [main] INFO backtype.storm.security.auth.AuthUtils - Got AutoCreds []
> Exception in thread "main" java.lang.RuntimeException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: JavaLoginConfig, provider: SUN, class: sun.security.provider.ConfigSpiFile)
> at backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:66)
> at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:82)
> at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:66)
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:52)
> at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:36)
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:211)
> at backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:273)
> at backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:254)
> at storm.starter.ExclamationTopology.main(ExclamationTopology.java:76)
> Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: JavaLoginConfig, provider: SUN, class: sun.security.provider.ConfigSpiFile)
> at java.security.Provider$Service.newInstance(Provider.java:1259)
> at sun.security.jca.GetInstance.getInstance(GetInstance.java:243)
> at sun.security.jca.GetInstance.getInstance(GetInstance.java:190)
> at javax.security.auth.login.Configuration.getInstance(Configuration.java:352)
> at backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:64)
> ... 8 more
> Caused by: java.io.IOException: Configuration Error:
> Line 1: expected [{], found [null]
> at com.sun.security.auth.login.ConfigFile.match(ConfigFile.java:572)
> at com.sun.security.auth.login.ConfigFile.parseLoginEntry(ConfigFile.java:405)
> at com.sun.security.auth.login.ConfigFile.readConfig(ConfigFile.java:383)
> at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:283)
> at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:166)
> at com.sun.security.auth.login.ConfigFile.<init>(ConfigFile.java:124)
> at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:72)
> at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:61)
> #
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.security.provider.ConfigSpiFile.<init>(ConfigSpiFile.java:61)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at java.security.Provider$Service.newInstance(Provider.java:1254)
> ... 12 more
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py 32cb60d
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py 5699e57
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2 cf78af4
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2 82ff239
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2 7c650df
>
> Diff: https://reviews.apache.org/r/27549/diff/
>
>
> Testing
> -------
>
> Ran unit tests on ambari-server,
> ----------------------------------------------------------------------
> Total run:684
> Total errors:0
> Total failures:0
> OK
>
>
> Reproduced the issue on a live cluster, and was able to get the topology added.
> E.g.,
> su - ambari-qa
> kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa@EXAMPLE.COM
>
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2 root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2 root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2 root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2 root@162.216.149.188:/tmp
>
>
> yes | cp /tmp/params.py /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/params.py
> yes | cp /tmp/storm.py /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/storm.py
> yes | cp /tmp/client_jaas.conf.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2
> yes | cp /tmp/storm.yaml.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2
> yes | cp /tmp/storm_jaas.conf.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2
> yes | cp /tmp/worker-launcher.cfg.j2 /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2
>
> ambari-server restart (to reload templates)
>
> Ensure that client_jaas.conf is in /etc/storm/conf/ . Then run,
> /usr/hdp/current/storm-client/bin/storm -c java.security.auth.login.config=/etc/storm/conf/client_jaas.conf jar /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1
>
>
> Thanks,
>
> Alejandro Fernandez
>
>