You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Nikos Koukos <pu...@gmail.com> on 2015/07/16 14:14:10 UTC

Deploying my topology to Remote Cluster (From: Eclipse, To: Cluster)

I have completed my topology in eclipse some days ago and i worked in
windows system, eclipse Luna with m2e 1.5 plugin. My to topology works fine
and now it is the time to deploy it to remote cluster.

So I export my project with eclipse as runnable JAR. (Right Click(on my
project)->Export...->Java->Runnable JAR file->Select radio button 'Package
required libraries into generated JAR file'->Finish

So i connect my pc to the server with putty, i upload my jar file with
WinSCP to the server and i run my topology:

-storm jar final.jar main.java.storm.Main submitted topology


But i get this exception:

Exception in thread "main" java.lang.ExceptionInInitializerError
>         at
> backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:340)
>         at
> backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:264)
>         at main.java.storm.Main.main(Main.java:47)
> Caused by: java.lang.RuntimeException: Found multiple defaults.yaml
> resources. You're probably bundling the Storm jars with your topology jar.
> [jar:file:/home/gdidymiotis/teliko_1.0.1_runnable.jar!/defaults.yaml,
> jar:file:/usr/hdp/2.2.4.2-2/storm/lib/storm-core-0.9.3.2.2.4.2-2.jar!/defaults.yaml]
>         at backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:139)
>         at backtype.storm.utils.Utils.readDefaultConfig(Utils.java:166)
>         at backtype.storm.utils.Utils.readStormConfig(Utils.java:190)
>         at backtype.storm.utils.Utils.<clinit>(Utils.java:77)
>         ... 3 more
>


In other post with similar problems people suggest to exclude some jar
files. But unfortunately there is not an option to exclude during export.
And also nobody describes another detailed solution with steps to follow.

Some extra maybe useful information i attach the pom.xml file(i really
don't know if this file make any difference in cluster mode) and a
print-screen of the extracted jar_file in order to see the files that are
inside the exported jar.

Re: Deploying my topology to Remote Cluster (From: Eclipse, To: Cluster)

Posted by Andrew Xor <an...@gmail.com>.
Well, that file contains all the dependencies that are required by your
topology as well as their versions. This can lead to a lot of hurdles not
only in your pom but to the server side as well. Maybe the storm version
installed in the remote cluster is different than the one you compiled and
bundled you topology against?

Also you stated that you topology works fine, does that means it just
'compiles' fine or you actually executed it? If you actually compiled and
executed it on the same Windows machine you mentioned then it has to be
noted that AFAIK storm is not (yet) officially supported in Windows...so
there's that...!
On Thu, Jul 16, 2015 at 6:44 PM Nikos Koukos <pu...@gmail.com>
wrote:

> Is it possible my pom.xml and its dependencies create all this trouble?
>
> 2015-07-16 16:29 GMT+03:00 Ziemer, Tom <to...@wirecard.com>:
>
>> Hi,
>>
>> I am using (maven)profiles for this:
>>
>>  <profile>
>>             <id>local</id>
>>             <activation>
>>                 <activeByDefault>true</activeByDefault>
>>             </activation>
>>             <properties>
>>                 <storm.core.scope>compile</storm.core.scope>
>>             </properties>
>>         </profile>
>>         <profile>
>>             <id>remote</id>
>>             <activation>
>>                 <activeByDefault>false</activeByDefault>
>>             </activation>
>>             <properties>
>>                 <storm.core.scope>provided</storm.core.scope>
>>             </properties>
>> </profile>
>>
>> And later:
>> <dependency>
>>             <groupId>org.apache.storm</groupId>
>>             <artifactId>storm-core</artifactId>
>>             <version>${storm.framework.version}</version>
>>             <!-- keep storm out of the jar-with-dependencies-->
>>             <scope>${storm.core.scope}</scope>
>>         </dependency>
>>
>> You can then create the artifact by running mvn clean install (local) or
>> mvn clean install -Premote for remote deployments.
>>
>> For creating a runnable jar, I recommend the shade-plugin:
>>
>> <build>
>>                 <plugins>
>>                     <plugin>
>>                         <groupId>org.apache.maven.plugins</groupId>
>>                         <artifactId>maven-shade-plugin</artifactId>
>>                         <version>2.3</version>
>>                         <executions>
>>                             <execution>
>>                                 <phase>package</phase>
>>                                 <goals>
>>                                     <goal>shade</goal>
>>                                 </goals>
>>                                 <configuration>
>>                                     <transformers>
>>                                         <transformer
>> implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
>>
>> <resource>META-INF/spring.handlers</resource>
>>                                         </transformer>
>>                                         <transformer
>> implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
>>
>> <resource>META-INF/spring.schemas</resource>
>>                                         </transformer>
>>                                     </transformers>
>>
>> <createDependencyReducedPom>false</createDependencyReducedPom>
>>                                 </configuration>
>>                             </execution>
>>                         </executions>
>>                     </plugin>
>>                 </plugins>
>>             </build>
>>
>> Regards,
>> Tom
>>
>> From: Alex Ott [mailto:alexott@gmail.com]
>> Sent: Donnerstag, 16. Juli 2015 14:53
>> To: user@storm.apache.org
>> Subject: Re: Deploying my topology to Remote Cluster (From: Eclipse, To:
>> Cluster)
>>
>> You can check the "Maven by Example" (
>> http://books.sonatype.com/mvnex-book/reference/public-book.html) first,
>> and continue with "Maven: The Complete Reference" (
>> http://books.sonatype.com/mvnref-book/reference/public-book.html) after
>> - these 2 books were parts of the "Maven: The Definitive Guide" by
>> O'Reilly...
>>
>> On Thu, Jul 16, 2015 at 2:25 PM, Nikos Koukos <pu...@gmail.com>
>> wrote:
>> No. I saw that a lot of people refer to this but i couldnt understand how
>> can i do this. I am a novice user of unix and storm. Can u guide me maybe
>> with some step-by-step help.
>> Thank you
>>
>> 2015-07-16 15:18 GMT+03:00 Alex Ott <al...@gmail.com>:
>> Have you tried to build it with maven using 'mvn assembly:assembly'
>> command?
>>
>> On Thu, Jul 16, 2015 at 2:14 PM, Nikos Koukos <pu...@gmail.com>
>> wrote:
>> I have completed my topology in eclipse some days ago and i worked in
>> windows system, eclipse Luna with m2e 1.5 plugin. My to topology works fine
>> and now it is the time to deploy it to remote cluster.
>> So I export my project with eclipse as runnable JAR. (Right Click(on my
>> project)->Export...->Java->Runnable JAR file->Select radio button 'Package
>> required libraries into generated JAR file'->Finish
>>
>> So i connect my pc to the server with putty, i upload my jar file with
>> WinSCP to the server and i run my topology:
>> -storm jar final.jar main.java.storm.Main submitted topology
>>
>> But i get this exception:
>> Exception in thread "main" java.lang.ExceptionInInitializerError
>>         at
>> backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:340)
>>         at
>> backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:264)
>>         at main.java.storm.Main.main(Main.java:47)
>> Caused by: java.lang.RuntimeException: Found multiple defaults.yaml
>> resources. You're probably bundling the Storm jars with your topology jar.
>> [jar:file:/home/gdidymiotis/teliko_1.0.1_runnable.jar!/defaults.yaml,
>> jar:file:/usr/hdp/2.2.4.2-2/storm/lib/storm-core-0.9.3.2.2.4.2-2.jar!/defaults.yaml]
>>         at
>> backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:139)
>>         at backtype.storm.utils.Utils.readDefaultConfig(Utils.java:166)
>>         at backtype.storm.utils.Utils.readStormConfig(Utils.java:190)
>>         at backtype.storm.utils.Utils.<clinit>(Utils.java:77)
>>         ... 3 more
>>
>> In other post with similar problems people suggest to exclude some jar
>> files. But unfortunately there is not an option to exclude during export.
>> And also nobody describes another detailed solution with steps to follow.
>> Some extra maybe useful information i attach the pom.xml file(i really
>> don't know if this file make any difference in cluster mode) and a
>> print-screen of the extracted jar_file in order to see the files that are
>> inside the exported jar.
>>
>>
>>
>> --
>> With best wishes,                    Alex Ott
>> http://alexott.net/
>> Twitter: alexott_en (English), alexott (Russian)
>> Skype: alex.ott
>>
>>
>>
>>
>> --
>> With best wishes,                    Alex Ott
>> http://alexott.net/
>> Twitter: alexott_en (English), alexott (Russian)
>> Skype: alex.ott
>>
>
>

Re: Deploying my topology to Remote Cluster (From: Eclipse, To: Cluster)

Posted by Nikos Koukos <pu...@gmail.com>.
Is it possible my pom.xml and its dependencies create all this trouble?

2015-07-16 16:29 GMT+03:00 Ziemer, Tom <to...@wirecard.com>:

> Hi,
>
> I am using (maven)profiles for this:
>
>  <profile>
>             <id>local</id>
>             <activation>
>                 <activeByDefault>true</activeByDefault>
>             </activation>
>             <properties>
>                 <storm.core.scope>compile</storm.core.scope>
>             </properties>
>         </profile>
>         <profile>
>             <id>remote</id>
>             <activation>
>                 <activeByDefault>false</activeByDefault>
>             </activation>
>             <properties>
>                 <storm.core.scope>provided</storm.core.scope>
>             </properties>
> </profile>
>
> And later:
> <dependency>
>             <groupId>org.apache.storm</groupId>
>             <artifactId>storm-core</artifactId>
>             <version>${storm.framework.version}</version>
>             <!-- keep storm out of the jar-with-dependencies-->
>             <scope>${storm.core.scope}</scope>
>         </dependency>
>
> You can then create the artifact by running mvn clean install (local) or
> mvn clean install -Premote for remote deployments.
>
> For creating a runnable jar, I recommend the shade-plugin:
>
> <build>
>                 <plugins>
>                     <plugin>
>                         <groupId>org.apache.maven.plugins</groupId>
>                         <artifactId>maven-shade-plugin</artifactId>
>                         <version>2.3</version>
>                         <executions>
>                             <execution>
>                                 <phase>package</phase>
>                                 <goals>
>                                     <goal>shade</goal>
>                                 </goals>
>                                 <configuration>
>                                     <transformers>
>                                         <transformer
> implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
>
> <resource>META-INF/spring.handlers</resource>
>                                         </transformer>
>                                         <transformer
> implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
>
> <resource>META-INF/spring.schemas</resource>
>                                         </transformer>
>                                     </transformers>
>
> <createDependencyReducedPom>false</createDependencyReducedPom>
>                                 </configuration>
>                             </execution>
>                         </executions>
>                     </plugin>
>                 </plugins>
>             </build>
>
> Regards,
> Tom
>
> From: Alex Ott [mailto:alexott@gmail.com]
> Sent: Donnerstag, 16. Juli 2015 14:53
> To: user@storm.apache.org
> Subject: Re: Deploying my topology to Remote Cluster (From: Eclipse, To:
> Cluster)
>
> You can check the "Maven by Example" (
> http://books.sonatype.com/mvnex-book/reference/public-book.html) first,
> and continue with "Maven: The Complete Reference" (
> http://books.sonatype.com/mvnref-book/reference/public-book.html) after -
> these 2 books were parts of the "Maven: The Definitive Guide" by O'Reilly...
>
> On Thu, Jul 16, 2015 at 2:25 PM, Nikos Koukos <pu...@gmail.com>
> wrote:
> No. I saw that a lot of people refer to this but i couldnt understand how
> can i do this. I am a novice user of unix and storm. Can u guide me maybe
> with some step-by-step help.
> Thank you
>
> 2015-07-16 15:18 GMT+03:00 Alex Ott <al...@gmail.com>:
> Have you tried to build it with maven using 'mvn assembly:assembly'
> command?
>
> On Thu, Jul 16, 2015 at 2:14 PM, Nikos Koukos <pu...@gmail.com>
> wrote:
> I have completed my topology in eclipse some days ago and i worked in
> windows system, eclipse Luna with m2e 1.5 plugin. My to topology works fine
> and now it is the time to deploy it to remote cluster.
> So I export my project with eclipse as runnable JAR. (Right Click(on my
> project)->Export...->Java->Runnable JAR file->Select radio button 'Package
> required libraries into generated JAR file'->Finish
>
> So i connect my pc to the server with putty, i upload my jar file with
> WinSCP to the server and i run my topology:
> -storm jar final.jar main.java.storm.Main submitted topology
>
> But i get this exception:
> Exception in thread "main" java.lang.ExceptionInInitializerError
>         at
> backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:340)
>         at
> backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:264)
>         at main.java.storm.Main.main(Main.java:47)
> Caused by: java.lang.RuntimeException: Found multiple defaults.yaml
> resources. You're probably bundling the Storm jars with your topology jar.
> [jar:file:/home/gdidymiotis/teliko_1.0.1_runnable.jar!/defaults.yaml,
> jar:file:/usr/hdp/2.2.4.2-2/storm/lib/storm-core-0.9.3.2.2.4.2-2.jar!/defaults.yaml]
>         at backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:139)
>         at backtype.storm.utils.Utils.readDefaultConfig(Utils.java:166)
>         at backtype.storm.utils.Utils.readStormConfig(Utils.java:190)
>         at backtype.storm.utils.Utils.<clinit>(Utils.java:77)
>         ... 3 more
>
> In other post with similar problems people suggest to exclude some jar
> files. But unfortunately there is not an option to exclude during export.
> And also nobody describes another detailed solution with steps to follow.
> Some extra maybe useful information i attach the pom.xml file(i really
> don't know if this file make any difference in cluster mode) and a
> print-screen of the extracted jar_file in order to see the files that are
> inside the exported jar.
>
>
>
> --
> With best wishes,                    Alex Ott
> http://alexott.net/
> Twitter: alexott_en (English), alexott (Russian)
> Skype: alex.ott
>
>
>
>
> --
> With best wishes,                    Alex Ott
> http://alexott.net/
> Twitter: alexott_en (English), alexott (Russian)
> Skype: alex.ott
>

RE: Deploying my topology to Remote Cluster (From: Eclipse, To: Cluster)

Posted by "Ziemer, Tom" <to...@wirecard.com>.
Hi,

I am using (maven)profiles for this:

 <profile>
            <id>local</id>
            <activation>
                <activeByDefault>true</activeByDefault>
            </activation>
            <properties>
                <storm.core.scope>compile</storm.core.scope>
            </properties>
        </profile>
        <profile>
            <id>remote</id>
            <activation>
                <activeByDefault>false</activeByDefault>
            </activation>
            <properties>
                <storm.core.scope>provided</storm.core.scope>
            </properties>
</profile>

And later:
<dependency>
            <groupId>org.apache.storm</groupId>
            <artifactId>storm-core</artifactId>
            <version>${storm.framework.version}</version>
            <!-- keep storm out of the jar-with-dependencies-->
            <scope>${storm.core.scope}</scope>
        </dependency>

You can then create the artifact by running mvn clean install (local) or mvn clean install -Premote for remote deployments.

For creating a runnable jar, I recommend the shade-plugin:

<build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-shade-plugin</artifactId>
                        <version>2.3</version>
                        <executions>
                            <execution>
                                <phase>package</phase>
                                <goals>
                                    <goal>shade</goal>
                                </goals>
                                <configuration>
                                    <transformers>
                                        <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                                            <resource>META-INF/spring.handlers</resource>
                                        </transformer>
                                        <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                                            <resource>META-INF/spring.schemas</resource>
                                        </transformer>
                                    </transformers>
                                    <createDependencyReducedPom>false</createDependencyReducedPom>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>

Regards,
Tom

From: Alex Ott [mailto:alexott@gmail.com] 
Sent: Donnerstag, 16. Juli 2015 14:53
To: user@storm.apache.org
Subject: Re: Deploying my topology to Remote Cluster (From: Eclipse, To: Cluster)

You can check the "Maven by Example" (http://books.sonatype.com/mvnex-book/reference/public-book.html) first, and continue with "Maven: The Complete Reference" (http://books.sonatype.com/mvnref-book/reference/public-book.html) after - these 2 books were parts of the "Maven: The Definitive Guide" by O'Reilly...

On Thu, Jul 16, 2015 at 2:25 PM, Nikos Koukos <pu...@gmail.com> wrote:
No. I saw that a lot of people refer to this but i couldnt understand how can i do this. I am a novice user of unix and storm. Can u guide me maybe with some step-by-step help. 
Thank you

2015-07-16 15:18 GMT+03:00 Alex Ott <al...@gmail.com>:
Have you tried to build it with maven using 'mvn assembly:assembly' command?

On Thu, Jul 16, 2015 at 2:14 PM, Nikos Koukos <pu...@gmail.com> wrote:
I have completed my topology in eclipse some days ago and i worked in windows system, eclipse Luna with m2e 1.5 plugin. My to topology works fine and now it is the time to deploy it to remote cluster. 
So I export my project with eclipse as runnable JAR. (Right Click(on my project)->Export...->Java->Runnable JAR file->Select radio button 'Package required libraries into generated JAR file'->Finish

So i connect my pc to the server with putty, i upload my jar file with WinSCP to the server and i run my topology: 
-storm jar final.jar main.java.storm.Main submitted topology

But i get this exception:
Exception in thread "main" java.lang.ExceptionInInitializerError
        at backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:340)
        at backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:264)
        at main.java.storm.Main.main(Main.java:47)
Caused by: java.lang.RuntimeException: Found multiple defaults.yaml resources. You're probably bundling the Storm jars with your topology jar. [jar:file:/home/gdidymiotis/teliko_1.0.1_runnable.jar!/defaults.yaml, jar:file:/usr/hdp/2.2.4.2-2/storm/lib/storm-core-0.9.3.2.2.4.2-2.jar!/defaults.yaml]
        at backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:139)
        at backtype.storm.utils.Utils.readDefaultConfig(Utils.java:166)
        at backtype.storm.utils.Utils.readStormConfig(Utils.java:190)
        at backtype.storm.utils.Utils.<clinit>(Utils.java:77)
        ... 3 more

In other post with similar problems people suggest to exclude some jar files. But unfortunately there is not an option to exclude during export. And also nobody describes another detailed solution with steps to follow. 
Some extra maybe useful information i attach the pom.xml file(i really don't know if this file make any difference in cluster mode) and a print-screen of the extracted jar_file in order to see the files that are inside the exported jar.



-- 
With best wishes,                    Alex Ott
http://alexott.net/
Twitter: alexott_en (English), alexott (Russian)
Skype: alex.ott




-- 
With best wishes,                    Alex Ott
http://alexott.net/
Twitter: alexott_en (English), alexott (Russian)
Skype: alex.ott

Re: Deploying my topology to Remote Cluster (From: Eclipse, To: Cluster)

Posted by Alex Ott <al...@gmail.com>.
You can check the "Maven by Example" (
http://books.sonatype.com/mvnex-book/reference/public-book.html) first, and
continue with "Maven: The Complete Reference" (
http://books.sonatype.com/mvnref-book/reference/public-book.html) after -
these 2 books were parts of the "Maven: The Definitive Guide" by O'Reilly...

On Thu, Jul 16, 2015 at 2:25 PM, Nikos Koukos <pu...@gmail.com>
wrote:

> No. I saw that a lot of people refer to this but i couldnt understand how
> can i do this. I am a novice user of unix and storm. Can u guide me maybe
> with some step-by-step help.
>
> Thank you
>
> 2015-07-16 15:18 GMT+03:00 Alex Ott <al...@gmail.com>:
>
>> Have you tried to build it with maven using 'mvn assembly:assembly'
>> command?
>>
>> On Thu, Jul 16, 2015 at 2:14 PM, Nikos Koukos <pu...@gmail.com>
>> wrote:
>>
>>> I have completed my topology in eclipse some days ago and i worked in
>>> windows system, eclipse Luna with m2e 1.5 plugin. My to topology works fine
>>> and now it is the time to deploy it to remote cluster.
>>>
>>> So I export my project with eclipse as runnable JAR. (Right Click(on my
>>> project)->Export...->Java->Runnable JAR file->Select radio button 'Package
>>> required libraries into generated JAR file'->Finish
>>>
>>> So i connect my pc to the server with putty, i upload my jar file with
>>> WinSCP to the server and i run my topology:
>>>
>>> -storm jar final.jar main.java.storm.Main submitted topology
>>>
>>>
>>> But i get this exception:
>>>
>>> Exception in thread "main" java.lang.ExceptionInInitializerError
>>>>         at
>>>> backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:340)
>>>>         at
>>>> backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:264)
>>>>         at main.java.storm.Main.main(Main.java:47)
>>>> Caused by: java.lang.RuntimeException: Found multiple defaults.yaml
>>>> resources. You're probably bundling the Storm jars with your topology jar.
>>>> [jar:file:/home/gdidymiotis/teliko_1.0.1_runnable.jar!/defaults.yaml,
>>>> jar:file:/usr/hdp/2.2.4.2-2/storm/lib/storm-core-0.9.3.2.2.4.2-2.jar!/defaults.yaml]
>>>>         at
>>>> backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:139)
>>>>         at backtype.storm.utils.Utils.readDefaultConfig(Utils.java:166)
>>>>         at backtype.storm.utils.Utils.readStormConfig(Utils.java:190)
>>>>         at backtype.storm.utils.Utils.<clinit>(Utils.java:77)
>>>>         ... 3 more
>>>>
>>>
>>>
>>> In other post with similar problems people suggest to exclude some jar
>>> files. But unfortunately there is not an option to exclude during export.
>>> And also nobody describes another detailed solution with steps to follow.
>>>
>>> Some extra maybe useful information i attach the pom.xml file(i really
>>> don't know if this file make any difference in cluster mode) and a
>>> print-screen of the extracted jar_file in order to see the files that are
>>> inside the exported jar.
>>>
>>
>>
>>
>> --
>> With best wishes,                    Alex Ott
>> http://alexott.net/
>> Twitter: alexott_en (English), alexott (Russian)
>> Skype: alex.ott
>>
>
>


-- 
With best wishes,                    Alex Ott
http://alexott.net/
Twitter: alexott_en (English), alexott (Russian)
Skype: alex.ott

Re: Deploying my topology to Remote Cluster (From: Eclipse, To: Cluster)

Posted by "Matthias J. Sax" <mj...@informatik.hu-berlin.de>.
Using Eclipe export will package "storm-core.jar" and it's dependencies
into you user jar. "storm-core.jar" contains the duplicate default.yaml
class. You should not include "storm-core.jar" and its dependencies
after all.

You might try "export -> Java -> jar". This only packages your own code.
If you don't have any depended libraries of you own code (expect storm
dependencies) this should work. Otherwise, you need to include those
manually into your jar (or deploy them on all node in you cluster in the
default storm-lib directory).

Using maven is a good alternative. However, it's more complicated as it
can be explained here in short. Please refer to the maven manual first.


-Matthias


On 07/16/2015 02:25 PM, Nikos Koukos wrote:
> No. I saw that a lot of people refer to this but i couldnt understand
> how can i do this. I am a novice user of unix and storm. Can u guide me
> maybe with some step-by-step help.
> 
> Thank you
> 
> 2015-07-16 15:18 GMT+03:00 Alex Ott <alexott@gmail.com
> <ma...@gmail.com>>:
> 
>     Have you tried to build it with maven using 'mvn assembly:assembly'
>     command?
> 
>     On Thu, Jul 16, 2015 at 2:14 PM, Nikos Koukos
>     <puteolana1902@gmail.com <ma...@gmail.com>> wrote:
> 
>         I have completed my topology in eclipse some days ago and i
>         worked in windows system, eclipse Luna with m2e 1.5 plugin. My
>         to topology works fine and now it is the time to deploy it to
>         remote cluster.
> 
>         So I export my project with eclipse as runnable JAR. (Right
>         Click(on my project)->Export...->Java->Runnable JAR file->Select
>         radio button 'Package required libraries into generated JAR
>         file'->Finish
> 
>         So i connect my pc to the server with putty, i upload my jar
>         file with WinSCP to the server and i run my topology:
> 
>             -storm jar final.jar main.java.storm.Main submitted topology
> 
> 
>         But i get this exception:
> 
>             Exception in thread "main" java.lang.ExceptionInInitializerError
>                     at
>             backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:340)
>                     at
>             backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:264)
>                     at main.java.storm.Main.main(Main.java:47)
>             Caused by: java.lang.RuntimeException: Found multiple
>             defaults.yaml resources. You're probably bundling the Storm
>             jars with your topology jar.
>             [jar:file:/home/gdidymiotis/teliko_1.0.1_runnable.jar!/defaults.yaml,
>             jar:file:/usr/hdp/2.2.4.2-2/storm/lib/storm-core-0.9.3.2.2.4.2-2.jar!/defaults.yaml]
>                     at
>             backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:139)
>                     at
>             backtype.storm.utils.Utils.readDefaultConfig(Utils.java:166)
>                     at
>             backtype.storm.utils.Utils.readStormConfig(Utils.java:190)
>                     at backtype.storm.utils.Utils.<clinit>(Utils.java:77)
>                     ... 3 more
> 
> 
> 
>         In other post with similar problems people suggest to exclude
>         some jar files. But unfortunately there is not an option to
>         exclude during export. And also nobody describes another
>         detailed solution with steps to follow.
> 
>         Some extra maybe useful information i attach the pom.xml file(i
>         really don't know if this file make any difference in cluster
>         mode) and a print-screen of the extracted jar_file in order to
>         see the files that are inside the exported jar.
> 
> 
> 
> 
>     -- 
>     With best wishes,                    Alex Ott
>     http://alexott.net/
>     Twitter: alexott_en (English), alexott (Russian)
>     Skype: alex.ott
> 
> 


Re: Deploying my topology to Remote Cluster (From: Eclipse, To: Cluster)

Posted by Nikos Koukos <pu...@gmail.com>.
No. I saw that a lot of people refer to this but i couldnt understand how
can i do this. I am a novice user of unix and storm. Can u guide me maybe
with some step-by-step help.

Thank you

2015-07-16 15:18 GMT+03:00 Alex Ott <al...@gmail.com>:

> Have you tried to build it with maven using 'mvn assembly:assembly'
> command?
>
> On Thu, Jul 16, 2015 at 2:14 PM, Nikos Koukos <pu...@gmail.com>
> wrote:
>
>> I have completed my topology in eclipse some days ago and i worked in
>> windows system, eclipse Luna with m2e 1.5 plugin. My to topology works fine
>> and now it is the time to deploy it to remote cluster.
>>
>> So I export my project with eclipse as runnable JAR. (Right Click(on my
>> project)->Export...->Java->Runnable JAR file->Select radio button 'Package
>> required libraries into generated JAR file'->Finish
>>
>> So i connect my pc to the server with putty, i upload my jar file with
>> WinSCP to the server and i run my topology:
>>
>> -storm jar final.jar main.java.storm.Main submitted topology
>>
>>
>> But i get this exception:
>>
>> Exception in thread "main" java.lang.ExceptionInInitializerError
>>>         at
>>> backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:340)
>>>         at
>>> backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:264)
>>>         at main.java.storm.Main.main(Main.java:47)
>>> Caused by: java.lang.RuntimeException: Found multiple defaults.yaml
>>> resources. You're probably bundling the Storm jars with your topology jar.
>>> [jar:file:/home/gdidymiotis/teliko_1.0.1_runnable.jar!/defaults.yaml,
>>> jar:file:/usr/hdp/2.2.4.2-2/storm/lib/storm-core-0.9.3.2.2.4.2-2.jar!/defaults.yaml]
>>>         at
>>> backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:139)
>>>         at backtype.storm.utils.Utils.readDefaultConfig(Utils.java:166)
>>>         at backtype.storm.utils.Utils.readStormConfig(Utils.java:190)
>>>         at backtype.storm.utils.Utils.<clinit>(Utils.java:77)
>>>         ... 3 more
>>>
>>
>>
>> In other post with similar problems people suggest to exclude some jar
>> files. But unfortunately there is not an option to exclude during export.
>> And also nobody describes another detailed solution with steps to follow.
>>
>> Some extra maybe useful information i attach the pom.xml file(i really
>> don't know if this file make any difference in cluster mode) and a
>> print-screen of the extracted jar_file in order to see the files that are
>> inside the exported jar.
>>
>
>
>
> --
> With best wishes,                    Alex Ott
> http://alexott.net/
> Twitter: alexott_en (English), alexott (Russian)
> Skype: alex.ott
>

Re: Deploying my topology to Remote Cluster (From: Eclipse, To: Cluster)

Posted by Alex Ott <al...@gmail.com>.
Have you tried to build it with maven using 'mvn assembly:assembly' command?

On Thu, Jul 16, 2015 at 2:14 PM, Nikos Koukos <pu...@gmail.com>
wrote:

> I have completed my topology in eclipse some days ago and i worked in
> windows system, eclipse Luna with m2e 1.5 plugin. My to topology works fine
> and now it is the time to deploy it to remote cluster.
>
> So I export my project with eclipse as runnable JAR. (Right Click(on my
> project)->Export...->Java->Runnable JAR file->Select radio button 'Package
> required libraries into generated JAR file'->Finish
>
> So i connect my pc to the server with putty, i upload my jar file with
> WinSCP to the server and i run my topology:
>
> -storm jar final.jar main.java.storm.Main submitted topology
>
>
> But i get this exception:
>
> Exception in thread "main" java.lang.ExceptionInInitializerError
>>         at
>> backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:340)
>>         at
>> backtype.storm.topology.TopologyBuilder$BoltGetter.customGrouping(TopologyBuilder.java:264)
>>         at main.java.storm.Main.main(Main.java:47)
>> Caused by: java.lang.RuntimeException: Found multiple defaults.yaml
>> resources. You're probably bundling the Storm jars with your topology jar.
>> [jar:file:/home/gdidymiotis/teliko_1.0.1_runnable.jar!/defaults.yaml,
>> jar:file:/usr/hdp/2.2.4.2-2/storm/lib/storm-core-0.9.3.2.2.4.2-2.jar!/defaults.yaml]
>>         at
>> backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:139)
>>         at backtype.storm.utils.Utils.readDefaultConfig(Utils.java:166)
>>         at backtype.storm.utils.Utils.readStormConfig(Utils.java:190)
>>         at backtype.storm.utils.Utils.<clinit>(Utils.java:77)
>>         ... 3 more
>>
>
>
> In other post with similar problems people suggest to exclude some jar
> files. But unfortunately there is not an option to exclude during export.
> And also nobody describes another detailed solution with steps to follow.
>
> Some extra maybe useful information i attach the pom.xml file(i really
> don't know if this file make any difference in cluster mode) and a
> print-screen of the extracted jar_file in order to see the files that are
> inside the exported jar.
>



-- 
With best wishes,                    Alex Ott
http://alexott.net/
Twitter: alexott_en (English), alexott (Russian)
Skype: alex.ott