You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by Ankur Garg <an...@gmail.com> on 2015/10/15 00:00:38 UTC
Topology not working + Found multiple defaults.yaml resources when
deploying to remote cluster
Hi ,
I have a single node remote cluster set up (all nimbus , supervisor ,
zookeeper) running on the same machine . I deployed my Topology (simple
Exclamation Topology) to this remote cluster . While topology and jar got
submitted successfully , nothing is happening in the cluster .
When I checked the supervisor logs , I could see this :
2015-10-14T21:24:26.340+0000 b.s.d.supervisor [INFO]
42dd0337-1182-45b0-9385-14570c7e0b09 still hasn't started
Worker log files are empty .
On debugging a little in supervisor logs , I could see
Launching worker with command: (Some java command) ..Firing this java
command I can see this error :
Caused by: java.lang.RuntimeException: Found multiple defaults.yaml
resources. You're probably bundling the Storm jars with your topology jar.
I debugged more on internet and other stuff , and modified my build.gradle
file too but still the same error whenever I deploy my topology .
This is my gradle file
dependencies {
compile group: 'org.springframework.boot', name:
'spring-boot-starter-actuator', version: springBootVersion
compile group: 'org.quartz-scheduler', name: 'quartz', version:
quartzVersion
compile group: 'clj-stacktrace' , name: 'clj-stacktrace',version:
cljStackTrace
compile group: 'org.apache.storm' , name: 'storm-core',version:
stormVersion
ext {
fatJarExclude = true
}
}
task uberjar(type: Jar) {
from files(sourceSets.main.output.classesDir)
from {configurations.compile.collect {zipTree(it)}} {
exclude "META-INF/*.SF"
exclude "META-INF/*.DSA"
exclude "META-INF/*.RSA"
exclude "META-INF/LICENSE"
}
manifest {
attributes 'Main-Class': 'storm.topology.ExclamationTopology'
}
}
Nybody has ny idea
Re: Topology not working + Found multiple defaults.yaml resources
when deploying to remote cluster
Posted by Ankur Garg <an...@gmail.com>.
Hi John ,
Thanks a lot for your reply . You pointed me in the right direction .
Here is a link that explains this
http://kennethjorgensen.com/blog/2014/fat-jars-with-excluded-dependencies-in-gradle
.
May be we shud add it to storm documentation somewhere if its not there so
that ppl dont end up wasting time .
Also , if possible we should fix this issue .
Thanks
Ankur
On Thu, Oct 15, 2015 at 3:47 AM, John Reilly <jr...@inconspicuous.org> wrote:
>
> I haven't tried this in gradle, but you basically need to do one of 2
> things.
>
> * Mark storm-core as provided
> or
> * Exclude storm-core.jar from the fat jar packaging.
>
> IIRC, you can change your storm dependency to
> provided group: 'org.apache.storm' ,
> name: 'storm-core',version: stormVersion
>
> I've tried doing this in my project (in sbt) and ran into the issue that
> when provided is used, the project appears to be broken in the IDE since
> the provided jar is not available in the IDE. Also, iirc if you try to run
> local cluster in the same project that will fail. Usually this is not a
> problem since it will usually be for a test where we can just specify
> storm-core as a test dependency.
>
> I ended up going with the second option above and as part of the
> packaging, I omit the storm-core jar from the fat jar. I'm not sure how
> you would do that in gradle though.
>
>
>
>
> On Wed, Oct 14, 2015 at 3:00 PM Ankur Garg <an...@gmail.com> wrote:
>
>> Hi ,
>>
>> I have a single node remote cluster set up (all nimbus , supervisor ,
>> zookeeper) running on the same machine . I deployed my Topology (simple
>> Exclamation Topology) to this remote cluster . While topology and jar got
>> submitted successfully , nothing is happening in the cluster .
>>
>> When I checked the supervisor logs , I could see this :
>>
>> 2015-10-14T21:24:26.340+0000 b.s.d.supervisor [INFO]
>> 42dd0337-1182-45b0-9385-14570c7e0b09 still hasn't started
>>
>> Worker log files are empty .
>>
>> On debugging a little in supervisor logs , I could see
>>
>> Launching worker with command: (Some java command) ..Firing this java
>> command I can see this error :
>>
>> Caused by: java.lang.RuntimeException: Found multiple defaults.yaml
>> resources. You're probably bundling the Storm jars with your topology jar.
>>
>> I debugged more on internet and other stuff , and modified my
>> build.gradle file too but still the same error whenever I deploy my
>> topology .
>>
>> This is my gradle file
>>
>>
>> dependencies {
>>
>> compile group: 'org.springframework.boot', name:
>> 'spring-boot-starter-actuator', version: springBootVersion
>>
>> compile group: 'org.quartz-scheduler', name: 'quartz', version:
>> quartzVersion
>>
>> compile group: 'clj-stacktrace' , name: 'clj-stacktrace',version:
>> cljStackTrace
>>
>> compile group: 'org.apache.storm' , name: 'storm-core',version:
>> stormVersion
>>
>> ext {
>>
>> fatJarExclude = true
>>
>> }
>>
>> }
>>
>>
>> task uberjar(type: Jar) {
>>
>> from files(sourceSets.main.output.classesDir)
>>
>> from {configurations.compile.collect {zipTree(it)}} {
>>
>> exclude "META-INF/*.SF"
>>
>> exclude "META-INF/*.DSA"
>>
>> exclude "META-INF/*.RSA"
>>
>> exclude "META-INF/LICENSE"
>>
>> }
>>
>>
>> manifest {
>>
>> attributes 'Main-Class': 'storm.topology.ExclamationTopology'
>>
>> }
>>
>> }
>>
>> Nybody has ny idea
>>
>
Re: Topology not working + Found multiple defaults.yaml resources
when deploying to remote cluster
Posted by Ankur Garg <an...@gmail.com>.
Hi John ,
Thanks a lot for your reply . You pointed me in the right direction .
Here is a link that explains this
http://kennethjorgensen.com/blog/2014/fat-jars-with-excluded-dependencies-in-gradle
.
May be we shud add it to storm documentation somewhere if its not there so
that ppl dont end up wasting time .
Also , if possible we should fix this issue .
Thanks
Ankur
On Thu, Oct 15, 2015 at 3:47 AM, John Reilly <jr...@inconspicuous.org> wrote:
>
> I haven't tried this in gradle, but you basically need to do one of 2
> things.
>
> * Mark storm-core as provided
> or
> * Exclude storm-core.jar from the fat jar packaging.
>
> IIRC, you can change your storm dependency to
> provided group: 'org.apache.storm' ,
> name: 'storm-core',version: stormVersion
>
> I've tried doing this in my project (in sbt) and ran into the issue that
> when provided is used, the project appears to be broken in the IDE since
> the provided jar is not available in the IDE. Also, iirc if you try to run
> local cluster in the same project that will fail. Usually this is not a
> problem since it will usually be for a test where we can just specify
> storm-core as a test dependency.
>
> I ended up going with the second option above and as part of the
> packaging, I omit the storm-core jar from the fat jar. I'm not sure how
> you would do that in gradle though.
>
>
>
>
> On Wed, Oct 14, 2015 at 3:00 PM Ankur Garg <an...@gmail.com> wrote:
>
>> Hi ,
>>
>> I have a single node remote cluster set up (all nimbus , supervisor ,
>> zookeeper) running on the same machine . I deployed my Topology (simple
>> Exclamation Topology) to this remote cluster . While topology and jar got
>> submitted successfully , nothing is happening in the cluster .
>>
>> When I checked the supervisor logs , I could see this :
>>
>> 2015-10-14T21:24:26.340+0000 b.s.d.supervisor [INFO]
>> 42dd0337-1182-45b0-9385-14570c7e0b09 still hasn't started
>>
>> Worker log files are empty .
>>
>> On debugging a little in supervisor logs , I could see
>>
>> Launching worker with command: (Some java command) ..Firing this java
>> command I can see this error :
>>
>> Caused by: java.lang.RuntimeException: Found multiple defaults.yaml
>> resources. You're probably bundling the Storm jars with your topology jar.
>>
>> I debugged more on internet and other stuff , and modified my
>> build.gradle file too but still the same error whenever I deploy my
>> topology .
>>
>> This is my gradle file
>>
>>
>> dependencies {
>>
>> compile group: 'org.springframework.boot', name:
>> 'spring-boot-starter-actuator', version: springBootVersion
>>
>> compile group: 'org.quartz-scheduler', name: 'quartz', version:
>> quartzVersion
>>
>> compile group: 'clj-stacktrace' , name: 'clj-stacktrace',version:
>> cljStackTrace
>>
>> compile group: 'org.apache.storm' , name: 'storm-core',version:
>> stormVersion
>>
>> ext {
>>
>> fatJarExclude = true
>>
>> }
>>
>> }
>>
>>
>> task uberjar(type: Jar) {
>>
>> from files(sourceSets.main.output.classesDir)
>>
>> from {configurations.compile.collect {zipTree(it)}} {
>>
>> exclude "META-INF/*.SF"
>>
>> exclude "META-INF/*.DSA"
>>
>> exclude "META-INF/*.RSA"
>>
>> exclude "META-INF/LICENSE"
>>
>> }
>>
>>
>> manifest {
>>
>> attributes 'Main-Class': 'storm.topology.ExclamationTopology'
>>
>> }
>>
>> }
>>
>> Nybody has ny idea
>>
>
Re: Topology not working + Found multiple defaults.yaml resources
when deploying to remote cluster
Posted by John Reilly <jr...@inconspicuous.org>.
I haven't tried this in gradle, but you basically need to do one of 2
things.
* Mark storm-core as provided
or
* Exclude storm-core.jar from the fat jar packaging.
IIRC, you can change your storm dependency to
provided group: 'org.apache.storm' ,
name: 'storm-core',version: stormVersion
I've tried doing this in my project (in sbt) and ran into the issue that
when provided is used, the project appears to be broken in the IDE since
the provided jar is not available in the IDE. Also, iirc if you try to run
local cluster in the same project that will fail. Usually this is not a
problem since it will usually be for a test where we can just specify
storm-core as a test dependency.
I ended up going with the second option above and as part of the packaging,
I omit the storm-core jar from the fat jar. I'm not sure how you would do
that in gradle though.
On Wed, Oct 14, 2015 at 3:00 PM Ankur Garg <an...@gmail.com> wrote:
> Hi ,
>
> I have a single node remote cluster set up (all nimbus , supervisor ,
> zookeeper) running on the same machine . I deployed my Topology (simple
> Exclamation Topology) to this remote cluster . While topology and jar got
> submitted successfully , nothing is happening in the cluster .
>
> When I checked the supervisor logs , I could see this :
>
> 2015-10-14T21:24:26.340+0000 b.s.d.supervisor [INFO]
> 42dd0337-1182-45b0-9385-14570c7e0b09 still hasn't started
>
> Worker log files are empty .
>
> On debugging a little in supervisor logs , I could see
>
> Launching worker with command: (Some java command) ..Firing this java
> command I can see this error :
>
> Caused by: java.lang.RuntimeException: Found multiple defaults.yaml
> resources. You're probably bundling the Storm jars with your topology jar.
>
> I debugged more on internet and other stuff , and modified my build.gradle
> file too but still the same error whenever I deploy my topology .
>
> This is my gradle file
>
>
> dependencies {
>
> compile group: 'org.springframework.boot', name:
> 'spring-boot-starter-actuator', version: springBootVersion
>
> compile group: 'org.quartz-scheduler', name: 'quartz', version:
> quartzVersion
>
> compile group: 'clj-stacktrace' , name: 'clj-stacktrace',version:
> cljStackTrace
>
> compile group: 'org.apache.storm' , name: 'storm-core',version:
> stormVersion
>
> ext {
>
> fatJarExclude = true
>
> }
>
> }
>
>
> task uberjar(type: Jar) {
>
> from files(sourceSets.main.output.classesDir)
>
> from {configurations.compile.collect {zipTree(it)}} {
>
> exclude "META-INF/*.SF"
>
> exclude "META-INF/*.DSA"
>
> exclude "META-INF/*.RSA"
>
> exclude "META-INF/LICENSE"
>
> }
>
>
> manifest {
>
> attributes 'Main-Class': 'storm.topology.ExclamationTopology'
>
> }
>
> }
>
> Nybody has ny idea
>
Re: Topology not working + Found multiple defaults.yaml resources
when deploying to remote cluster
Posted by John Reilly <jr...@inconspicuous.org>.
I haven't tried this in gradle, but you basically need to do one of 2
things.
* Mark storm-core as provided
or
* Exclude storm-core.jar from the fat jar packaging.
IIRC, you can change your storm dependency to
provided group: 'org.apache.storm' ,
name: 'storm-core',version: stormVersion
I've tried doing this in my project (in sbt) and ran into the issue that
when provided is used, the project appears to be broken in the IDE since
the provided jar is not available in the IDE. Also, iirc if you try to run
local cluster in the same project that will fail. Usually this is not a
problem since it will usually be for a test where we can just specify
storm-core as a test dependency.
I ended up going with the second option above and as part of the packaging,
I omit the storm-core jar from the fat jar. I'm not sure how you would do
that in gradle though.
On Wed, Oct 14, 2015 at 3:00 PM Ankur Garg <an...@gmail.com> wrote:
> Hi ,
>
> I have a single node remote cluster set up (all nimbus , supervisor ,
> zookeeper) running on the same machine . I deployed my Topology (simple
> Exclamation Topology) to this remote cluster . While topology and jar got
> submitted successfully , nothing is happening in the cluster .
>
> When I checked the supervisor logs , I could see this :
>
> 2015-10-14T21:24:26.340+0000 b.s.d.supervisor [INFO]
> 42dd0337-1182-45b0-9385-14570c7e0b09 still hasn't started
>
> Worker log files are empty .
>
> On debugging a little in supervisor logs , I could see
>
> Launching worker with command: (Some java command) ..Firing this java
> command I can see this error :
>
> Caused by: java.lang.RuntimeException: Found multiple defaults.yaml
> resources. You're probably bundling the Storm jars with your topology jar.
>
> I debugged more on internet and other stuff , and modified my build.gradle
> file too but still the same error whenever I deploy my topology .
>
> This is my gradle file
>
>
> dependencies {
>
> compile group: 'org.springframework.boot', name:
> 'spring-boot-starter-actuator', version: springBootVersion
>
> compile group: 'org.quartz-scheduler', name: 'quartz', version:
> quartzVersion
>
> compile group: 'clj-stacktrace' , name: 'clj-stacktrace',version:
> cljStackTrace
>
> compile group: 'org.apache.storm' , name: 'storm-core',version:
> stormVersion
>
> ext {
>
> fatJarExclude = true
>
> }
>
> }
>
>
> task uberjar(type: Jar) {
>
> from files(sourceSets.main.output.classesDir)
>
> from {configurations.compile.collect {zipTree(it)}} {
>
> exclude "META-INF/*.SF"
>
> exclude "META-INF/*.DSA"
>
> exclude "META-INF/*.RSA"
>
> exclude "META-INF/LICENSE"
>
> }
>
>
> manifest {
>
> attributes 'Main-Class': 'storm.topology.ExclamationTopology'
>
> }
>
> }
>
> Nybody has ny idea
>