You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by shahab <sh...@gmail.com> on 2014/05/05 16:41:43 UTC
Order of Bolt definition, catching ""that subscribes from
non-existent component [ ...]"
Hi,
I am trying to define a topology as following:
S : is a spout
A,B,C : are bolts
--> : means emitting message
S -->A
A -->B
B -->C
C -->A
I am declaring the Spouts and Bolts in the above order in my java code ,
first S, then A , B and finally C.
I am using globalGrouping("BoltName", StreamID) for collecting messages to
be collected by each bolt,
The problem is that I receive an error, while defining bolt "A" saying
"that subscribes from non-existent component [C]" .
I guess the error is happening because component "C" is not defined yet!
but what could be the solution to this?
best,
/Shahab
Re: Order of Bolt definition, catching ""that subscribes from
non-existent component [ ...]"
Posted by Michael Rose <mi...@fullcontact.com>.
You can have a loop on a different stream. It's not always the best thing
to do (deadlock possibilities from buffers) but we have a production
topology that has that kind of pattern. In our case, one bolt acts as a
coordinator for recursive search.
Michael Rose (@Xorlev <https://twitter.com/xorlev>)
Senior Platform Engineer, FullContact <http://www.fullcontact.com/>
michael@fullcontact.com
On Fri, Jun 6, 2014 at 2:28 PM, Abhishek Bhattacharjee <
abhishek.bhattacharjee11@gmail.com> wrote:
> I am sorry for the late reply.
> Yes , you can't have a loop. You can have a chain though( which doesn't
> close upon itself ! ).
>
> Thanks :-)
>
>
>
> On Wed, May 7, 2014 at 12:50 PM, shahab <sh...@gmail.com> wrote:
>
>> Thanks Abhishek. But this also implies that we can not have a loop ( of
>> message processing stages) using Storm, right?
>>
>> best,
>> /Shahab
>>
>>
>> On Mon, May 5, 2014 at 9:45 PM, Abhishek Bhattacharjee <
>> abhishek.bhattacharjee11@gmail.com> wrote:
>>
>>> I don't think what you are trying to do is achievable. Data in storm
>>> always move forward so you can't give it back to a bolt from which it
>>> originated. That is a bolt can subscribe from bolts which were created
>>> before it's creation. So, I think you can create another object of the A
>>> bolt say D and then assign the o/p of C to D.
>>>
>>>
>>> On Mon, May 5, 2014 at 8:11 PM, shahab <sh...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am trying to define a topology as following:
>>>> S : is a spout
>>>> A,B,C : are bolts
>>>> --> : means emitting message
>>>>
>>>> S -->A
>>>> A -->B
>>>> B -->C
>>>> C -->A
>>>>
>>>> I am declaring the Spouts and Bolts in the above order in my java code
>>>> , first S, then A , B and finally C.
>>>>
>>>> I am using globalGrouping("BoltName", StreamID) for collecting
>>>> messages to be collected by each bolt,
>>>>
>>>> The problem is that I receive an error, while defining bolt "A" saying
>>>> "that subscribes from non-existent component [C]" .
>>>>
>>>> I guess the error is happening because component "C" is not defined yet!
>>>> but what could be the solution to this?
>>>>
>>>> best,
>>>> /Shahab
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Abhishek Bhattacharjee*
>>> *Pune Institute of Computer Technology*
>>>
>>
>>
>
>
> --
> *Abhishek Bhattacharjee*
> *Pune Institute of Computer Technology*
>
Re: Order of Bolt definition, catching ""that subscribes from
non-existent component [ ...]"
Posted by Abhishek Bhattacharjee <ab...@gmail.com>.
I am sorry for the late reply.
Yes , you can't have a loop. You can have a chain though( which doesn't
close upon itself ! ).
Thanks :-)
On Wed, May 7, 2014 at 12:50 PM, shahab <sh...@gmail.com> wrote:
> Thanks Abhishek. But this also implies that we can not have a loop ( of
> message processing stages) using Storm, right?
>
> best,
> /Shahab
>
>
> On Mon, May 5, 2014 at 9:45 PM, Abhishek Bhattacharjee <
> abhishek.bhattacharjee11@gmail.com> wrote:
>
>> I don't think what you are trying to do is achievable. Data in storm
>> always move forward so you can't give it back to a bolt from which it
>> originated. That is a bolt can subscribe from bolts which were created
>> before it's creation. So, I think you can create another object of the A
>> bolt say D and then assign the o/p of C to D.
>>
>>
>> On Mon, May 5, 2014 at 8:11 PM, shahab <sh...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I am trying to define a topology as following:
>>> S : is a spout
>>> A,B,C : are bolts
>>> --> : means emitting message
>>>
>>> S -->A
>>> A -->B
>>> B -->C
>>> C -->A
>>>
>>> I am declaring the Spouts and Bolts in the above order in my java code ,
>>> first S, then A , B and finally C.
>>>
>>> I am using globalGrouping("BoltName", StreamID) for collecting messages
>>> to be collected by each bolt,
>>>
>>> The problem is that I receive an error, while defining bolt "A" saying
>>> "that subscribes from non-existent component [C]" .
>>>
>>> I guess the error is happening because component "C" is not defined yet!
>>> but what could be the solution to this?
>>>
>>> best,
>>> /Shahab
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>> *Abhishek Bhattacharjee*
>> *Pune Institute of Computer Technology*
>>
>
>
--
*Abhishek Bhattacharjee*
*Pune Institute of Computer Technology*
Re: Order of Bolt definition, catching ""that subscribes from
non-existent component [ ...]"
Posted by shahab <sh...@gmail.com>.
Thanks Abhishek. But this also implies that we can not have a loop ( of
message processing stages) using Storm, right?
best,
/Shahab
On Mon, May 5, 2014 at 9:45 PM, Abhishek Bhattacharjee <
abhishek.bhattacharjee11@gmail.com> wrote:
> I don't think what you are trying to do is achievable. Data in storm
> always move forward so you can't give it back to a bolt from which it
> originated. That is a bolt can subscribe from bolts which were created
> before it's creation. So, I think you can create another object of the A
> bolt say D and then assign the o/p of C to D.
>
>
> On Mon, May 5, 2014 at 8:11 PM, shahab <sh...@gmail.com> wrote:
>
>> Hi,
>>
>> I am trying to define a topology as following:
>> S : is a spout
>> A,B,C : are bolts
>> --> : means emitting message
>>
>> S -->A
>> A -->B
>> B -->C
>> C -->A
>>
>> I am declaring the Spouts and Bolts in the above order in my java code ,
>> first S, then A , B and finally C.
>>
>> I am using globalGrouping("BoltName", StreamID) for collecting messages
>> to be collected by each bolt,
>>
>> The problem is that I receive an error, while defining bolt "A" saying
>> "that subscribes from non-existent component [C]" .
>>
>> I guess the error is happening because component "C" is not defined yet!
>> but what could be the solution to this?
>>
>> best,
>> /Shahab
>>
>>
>>
>>
>>
>>
>>
>
>
> --
> *Abhishek Bhattacharjee*
> *Pune Institute of Computer Technology*
>
Re: Order of Bolt definition, catching ""that subscribes from
non-existent component [ ...]"
Posted by Abhishek Bhattacharjee <ab...@gmail.com>.
I don't think what you are trying to do is achievable. Data in storm always
move forward so you can't give it back to a bolt from which it originated.
That is a bolt can subscribe from bolts which were created before it's
creation. So, I think you can create another object of the A bolt say D and
then assign the o/p of C to D.
On Mon, May 5, 2014 at 8:11 PM, shahab <sh...@gmail.com> wrote:
> Hi,
>
> I am trying to define a topology as following:
> S : is a spout
> A,B,C : are bolts
> --> : means emitting message
>
> S -->A
> A -->B
> B -->C
> C -->A
>
> I am declaring the Spouts and Bolts in the above order in my java code ,
> first S, then A , B and finally C.
>
> I am using globalGrouping("BoltName", StreamID) for collecting messages
> to be collected by each bolt,
>
> The problem is that I receive an error, while defining bolt "A" saying
> "that subscribes from non-existent component [C]" .
>
> I guess the error is happening because component "C" is not defined yet!
> but what could be the solution to this?
>
> best,
> /Shahab
>
>
>
>
>
>
>
--
*Abhishek Bhattacharjee*
*Pune Institute of Computer Technology*