You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Vineet Mishra <cl...@gmail.com> on 2016/03/29 21:58:40 UTC
Best way to Submit a Job to Storm Topology
Hi All,
Its been long since I have been not in touch with Storm and was looking out
some standard way to submit a job/topology to the Storm.
I was building the dependencies(using Kafka Spout as Input) using Maven but
then landing up with duplicate issue while running the topology.
Any leads would be appreciated.
Thanks!
VM
Re: 答复: Best way to Submit a Job to Storm Topology
Posted by Abhishek Agarwal <ab...@gmail.com>.
You need to exclude storm-core from your uber jar.
On Wed, Mar 30, 2016 at 11:51 AM, Erik Weathers <ew...@groupon.com>
wrote:
> Can you please be more specific? i.e., which jars are considered
> duplicate? Can you provide a stack trace?
> What version of storm are you using? There have been a lot of changes
> regarding storm-core's shading of internal dependencies.
>
> - Erik
>
> On Tue, Mar 29, 2016 at 10:34 PM, Vineet Mishra <cl...@gmail.com>
> wrote:
>
>> Hi Sai and All,
>>
>> What I meant by duplicate issue was that while running the jar through
>> StormSubmitter (which was build through maven) I got duplicate jars present
>> for the Storm which was failing at the job submission.
>>
>> Any workaround or standard practice to avoid the same.
>>
>> Thanks!
>> VM
>> On Mar 30, 2016 9:43 AM, "Sai Dilip Reddy Kiralam" <
>> dkiralam@aadhya-analytics.com> wrote:
>>
>>> Hi vineet,
>>>
>>> If I understand you correctly your saying that ' when you submitted the
>>> topology you noticed that your kafka spout is getting some duplicates and
>>> sending it storm topology for processing'. Right?
>>>
>>>
>>> *Best regards,*
>>> *K.Sai Dilip Reddy,*
>>>
>>>
>>>
>>>>
>>>
>
--
Regards,
Abhishek Agarwal
Re: 答复: Best way to Submit a Job to Storm Topology
Posted by Erik Weathers <ew...@groupon.com>.
Can you please be more specific? i.e., which jars are considered
duplicate? Can you provide a stack trace?
What version of storm are you using? There have been a lot of changes
regarding storm-core's shading of internal dependencies.
- Erik
On Tue, Mar 29, 2016 at 10:34 PM, Vineet Mishra <cl...@gmail.com>
wrote:
> Hi Sai and All,
>
> What I meant by duplicate issue was that while running the jar through
> StormSubmitter (which was build through maven) I got duplicate jars present
> for the Storm which was failing at the job submission.
>
> Any workaround or standard practice to avoid the same.
>
> Thanks!
> VM
> On Mar 30, 2016 9:43 AM, "Sai Dilip Reddy Kiralam" <
> dkiralam@aadhya-analytics.com> wrote:
>
>> Hi vineet,
>>
>> If I understand you correctly your saying that ' when you submitted the
>> topology you noticed that your kafka spout is getting some duplicates and
>> sending it storm topology for processing'. Right?
>>
>>
>> *Best regards,*
>> *K.Sai Dilip Reddy,*
>>
>>
>>
>>>
>>
Re: 答复: Best way to Submit a Job to Storm Topology
Posted by Vineet Mishra <cl...@gmail.com>.
Hi Sai and All,
What I meant by duplicate issue was that while running the jar through
StormSubmitter (which was build through maven) I got duplicate jars present
for the Storm which was failing at the job submission.
Any workaround or standard practice to avoid the same.
Thanks!
VM
On Mar 30, 2016 9:43 AM, "Sai Dilip Reddy Kiralam" <
dkiralam@aadhya-analytics.com> wrote:
> Hi vineet,
>
> If I understand you correctly your saying that ' when you submitted the
> topology you noticed that your kafka spout is getting some duplicates and
> sending it storm topology for processing'. Right?
>
>
> *Best regards,*
> *K.Sai Dilip Reddy,*
>
>
>
>>
>
Re: 答复: Best way to Submit a Job to Storm Topology
Posted by Sai Dilip Reddy Kiralam <dk...@aadhya-analytics.com>.
Hi vineet,
If I understand you correctly your saying that ' when you submitted the
topology you noticed that your kafka spout is getting some duplicates and
sending it storm topology for processing'. Right?
*Best regards,*
*K.Sai Dilip Reddy,*
>
Re: 答复: Best way to Submit a Job to Storm Topology
Posted by Andrew Xor <an...@gmail.com>.
Hello,
I don't know if that fits the your case but the standard way of submitting
a storm topology AFAIK is the StormSubmitter class. The class source is
located here
<https://github.com/apache/storm/blob/a4f9f8bc5b4ca85de487a0a868e519ddcb94e852/storm-core/src/jvm/org/apache/storm/StormSubmitter.java>
and a simple example here
<https://github.com/apache/storm/blob/a4f9f8bc5b4ca85de487a0a868e519ddcb94e852/examples/storm-starter/src/jvm/org/apache/storm/starter/util/StormRunner.java>
.
A.
On Wed, Mar 30, 2016 at 5:14 AM, John Fang <xi...@alibaba-inc.com>
wrote:
> What duplicate issue?
>
>
>
> *发件人:* Vineet Mishra [mailto:clearmidoubt@gmail.com]
> *发送时间:* 2016年3月30日 3:59
> *收件人:* user@storm.apache.org
> *主题:* Best way to Submit a Job to Storm Topology
>
>
>
> Hi All,
>
>
>
> Its been long since I have been not in touch with Storm and was looking
> out some standard way to submit a job/topology to the Storm.
>
>
>
> I was building the dependencies(using Kafka Spout as Input) using Maven
> but then landing up with duplicate issue while running the topology.
>
>
>
> Any leads would be appreciated.
>
>
>
> Thanks!
>
> VM
>
答复: Best way to Submit a Job to Storm Topology
Posted by John Fang <xi...@alibaba-inc.com>.
What duplicate issue?
发件人: Vineet Mishra [mailto:clearmidoubt@gmail.com]
发送时间: 2016年3月30日 3:59
收件人: user@storm.apache.org
主题: Best way to Submit a Job to Storm Topology
Hi All,
Its been long since I have been not in touch with Storm and was looking out some standard way to submit a job/topology to the Storm.
I was building the dependencies(using Kafka Spout as Input) using Maven but then landing up with duplicate issue while running the topology.
Any leads would be appreciated.
Thanks!
VM