You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Alexander T <mi...@gmail.com> on 2016/03/04 15:34:58 UTC
Stateful bolts and acking
Hello!
I'm on the 1.x-branch and trying to use the stateful bolts. But I cannot
get it to work with acking. No matter what I try acking simply doesn't seem
to have any effect. I see acking tasks suceeding in the logs (ack tuples
forwarded to the acker, acker runs successfully), but still acking is not
succesful and the tuples fail after the message timeout. I've not been
successful in debugging the acker and the rotating map to figure out why
acking is not completing. I was wondering if acking is supposed to work
with stateful bolts, if so how and if there are any examples? Any ideas
would be appreciated.
Best regards,
Alexander T
Re: Stateful bolts and acking
Posted by Alexander T <mi...@gmail.com>.
Hi Arun,
I'm extending from BaseStatefulBolt<KeyValueState<String, String>, only
empty impls and I'm anchoring in the spout emit. And I'm running in
LocalCluster. I also tried with premade spouts to assure that I didn't make
a mistake there. Which branch should I try? Is head of the 1.x-branch the
right one? Maybe I'm on an old branch or unstable? Which version are you
testing against?
If I can't get it to work I'll zip up a testcase and paste it here, maybe
you'll spot something... Must be something silly.
Cheers
On Mar 4, 2016 7:26 PM, "Arun Mahadevan" <ar...@apache.org> wrote:
> Hi Alexander,
>
> The simple topology like the one you have mentioned with a single spout
> and a stateful bolt with an empty execute works for me. I assume you are
> anchoring the tuple in the spout’s emit.
>
> For a stateful topology, the acking doesn’t work if a bolt extends from
> BaseBasicBolt or if theres an “ack" call in the bolt’s execute.
>
> You could open a JIRA (https://issues.apache.org/jira/browse/STORM) and
> share the sample topology code where you see this issue.
>
> Thanks,
> Arun
>
>
> From: Alexander T
> Reply-To: "user@storm.apache.org"
> Date: Friday, March 4, 2016 at 11:15 PM
> To: "user@storm.apache.org"
> Subject: Re: Stateful bolts and acking
>
> Hi Arun,
>
> Thank you for your reply. I have tried both manual and automatic acking,
> and neither works. Or rather, the acker is running but my spout is still
> receiving failure callbacks after the message timeout. I have a simple
> topology with a single spout reading from a file and a single empty bolt
> doing nothing. If I change from an empty stateless bolt to an empty
> stateful bolt the acking stops working.
>
> Should I proceed with sending this in as a reproducing test case? Or is
> there anything else which I might have missed?
>
> Regards,
> Alexander
> On Mar 4, 2016 5:30 PM, "Arun Mahadevan" <ar...@apache.org> wrote:
>
>> Hi Alexander,
>>
>> For a stateful topology the anchoring and acking is automatically taken
>> care of.
>>
>> Can you check if any of your bolts inherit BaseBasicBolt or if you are
>> manually acking. Your non-stateful bolts could inherit from BaseRichBolt
>> instead.
>>
>> Thanks,
>> Arun
>>
>>
>> From: Alexander T
>> Reply-To: "user@storm.apache.org"
>> Date: Friday, March 4, 2016 at 8:04 PM
>> To: "user@storm.apache.org"
>> Subject: Stateful bolts and acking
>>
>> Hello!
>>
>> I'm on the 1.x-branch and trying to use the stateful bolts. But I cannot
>> get it to work with acking. No matter what I try acking simply doesn't seem
>> to have any effect. I see acking tasks suceeding in the logs (ack tuples
>> forwarded to the acker, acker runs successfully), but still acking is not
>> succesful and the tuples fail after the message timeout. I've not been
>> successful in debugging the acker and the rotating map to figure out why
>> acking is not completing. I was wondering if acking is supposed to work
>> with stateful bolts, if so how and if there are any examples? Any ideas
>> would be appreciated.
>>
>> Best regards,
>>
>> Alexander T
>>
>
Re: Stateful bolts and acking
Posted by Alexander T <mi...@gmail.com>.
Hi Arun,
That is a correct understanding. Our use-case is that we cannot live up to
the total ordering requirement of the incoming tuples to a stateful
windowed bolt due to performance reasons. Ideally we would like to
construct our own stateful windowed bolt which would not try to replay the
last window in a strict order, but simply drop that data (acceptable to us)
or replay it in arbitrary order. Manual acking sounds good since we can
probably implement this on our own then. As it is now, if I understand you
corrently, there is no way of delaying acking in a stateless bolt due to
the auto-acking feature. Great to hear that the patch is under way. Thanks
a lot!
Best regards
Alexander
On Tue, Mar 8, 2016 at 9:58 AM, Arun Mahadevan <ar...@apache.org> wrote:
> Hi Alexander,
>
> If I understand correctly, you have a stateful bolt and a windowed bolt
> that extends BaseWindowedBolt in your topology and the tuples are acked
> before the execute method of the windowed bolt is invoked. I think this is
> because we auto ack the non-stateful bolt (windowed bolt) tuples. This
> should get fixed with the patch to remove auto acking for non stateful
> bolts.
>
> Thanks,
> Arun
>
> From: Alexander T
> Reply-To: "user@storm.apache.org"
> Date: Tuesday, March 8, 2016 at 1:55 PM
>
> To: "user@storm.apache.org"
> Subject: Re: Stateful bolts and acking
>
> Hi Arun,
>
> Great to hear about the patch you are working on. I have another question
> if you have time: Right now I'm seeing acking a bit early in the chain,
> specifically for a windowed bolt we have which extends BaseWindowedBolt.
> The messages seem to be acked right before they reach this bolt, and if I
> replace it with a simple stateful bolt, the acking starts working as
> expected again. Is there support for non-stateful windowed bolts?
>
> Best regards
>
> Alexander
>
> On Mon, Mar 7, 2016 at 12:16 PM, Arun Mahadevan <ar...@apache.org> wrote:
>
>> Hi Alexander,
>>
>> You are right, the acking for the non-stateful bolts in a stateful
>> topology is better handled by the bolts themselves.
>>
>> I plan to submit a patch soon. You can track it here -
>> https://issues.apache.org/jira/browse/STORM-1608
>>
>> Thanks,
>> Arun
>>
>> From: Alexander T
>> Reply-To: "user@storm.apache.org"
>> Date: Monday, March 7, 2016 at 4:37 PM
>>
>> To: "user@storm.apache.org"
>> Subject: Re: Stateful bolts and acking
>>
>> Hi Arun,
>>
>> After reading your messages a bit more thoroughly (sorry :) ) I managed
>> to get the basic case working. If I understand you correctly, one should
>> inherit from BaseRichBolt, which normally requires manual acking, but *not*
>> perform any manual acking. And one should *not* extends BaseBasicBolt which
>> normally auto-acks, since this auto-acking will not work as intended with
>> stateful topologies.
>>
>> This would unfortunately mean that we cannot use bolts which were
>> written for stateless topologies in stateful ones. Are there any plans of
>> adapters or to change the interface so that they can interoperate?
>>
>> Best regards
>> Alexander
>>
>> On Fri, Mar 4, 2016 at 7:26 PM, Arun Mahadevan <ar...@apache.org> wrote:
>>
>>> Hi Alexander,
>>>
>>> The simple topology like the one you have mentioned with a single spout
>>> and a stateful bolt with an empty execute works for me. I assume you are
>>> anchoring the tuple in the spout’s emit.
>>>
>>> For a stateful topology, the acking doesn’t work if a bolt extends from
>>> BaseBasicBolt or if theres an “ack" call in the bolt’s execute.
>>>
>>> You could open a JIRA (https://issues.apache.org/jira/browse/STORM) and
>>> share the sample topology code where you see this issue.
>>>
>>> Thanks,
>>> Arun
>>>
>>>
>>> From: Alexander T
>>> Reply-To: "user@storm.apache.org"
>>> Date: Friday, March 4, 2016 at 11:15 PM
>>> To: "user@storm.apache.org"
>>> Subject: Re: Stateful bolts and acking
>>>
>>> Hi Arun,
>>>
>>> Thank you for your reply. I have tried both manual and automatic acking,
>>> and neither works. Or rather, the acker is running but my spout is still
>>> receiving failure callbacks after the message timeout. I have a simple
>>> topology with a single spout reading from a file and a single empty bolt
>>> doing nothing. If I change from an empty stateless bolt to an empty
>>> stateful bolt the acking stops working.
>>>
>>> Should I proceed with sending this in as a reproducing test case? Or is
>>> there anything else which I might have missed?
>>>
>>> Regards,
>>> Alexander
>>> On Mar 4, 2016 5:30 PM, "Arun Mahadevan" <ar...@apache.org> wrote:
>>>
>>>> Hi Alexander,
>>>>
>>>> For a stateful topology the anchoring and acking is automatically taken
>>>> care of.
>>>>
>>>> Can you check if any of your bolts inherit BaseBasicBolt or if you are
>>>> manually acking. Your non-stateful bolts could inherit from BaseRichBolt
>>>> instead.
>>>>
>>>> Thanks,
>>>> Arun
>>>>
>>>>
>>>> From: Alexander T
>>>> Reply-To: "user@storm.apache.org"
>>>> Date: Friday, March 4, 2016 at 8:04 PM
>>>> To: "user@storm.apache.org"
>>>> Subject: Stateful bolts and acking
>>>>
>>>> Hello!
>>>>
>>>> I'm on the 1.x-branch and trying to use the stateful bolts. But I
>>>> cannot get it to work with acking. No matter what I try acking simply
>>>> doesn't seem to have any effect. I see acking tasks suceeding in the logs
>>>> (ack tuples forwarded to the acker, acker runs successfully), but still
>>>> acking is not succesful and the tuples fail after the message timeout. I've
>>>> not been successful in debugging the acker and the rotating map to figure
>>>> out why acking is not completing. I was wondering if acking is supposed to
>>>> work with stateful bolts, if so how and if there are any examples? Any
>>>> ideas would be appreciated.
>>>>
>>>> Best regards,
>>>>
>>>> Alexander T
>>>>
>>>
>>
>
Re: Stateful bolts and acking
Posted by Arun Mahadevan <ar...@apache.org>.
Hi Alexander,
If I understand correctly, you have a stateful bolt and a windowed bolt that extends BaseWindowedBolt in your topology and the tuples are acked before the execute method of the windowed bolt is invoked. I think this is because we auto ack the non-stateful bolt (windowed bolt) tuples. This should get fixed with the patch to remove auto acking for non stateful bolts.
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Tuesday, March 8, 2016 at 1:55 PM
To: "user@storm.apache.org"
Subject: Re: Stateful bolts and acking
Hi Arun,
Great to hear about the patch you are working on. I have another question if you have time: Right now I'm seeing acking a bit early in the chain, specifically for a windowed bolt we have which extends BaseWindowedBolt. The messages seem to be acked right before they reach this bolt, and if I replace it with a simple stateful bolt, the acking starts working as expected again. Is there support for non-stateful windowed bolts?
Best regards
Alexander
On Mon, Mar 7, 2016 at 12:16 PM, Arun Mahadevan <ar...@apache.org> wrote:
Hi Alexander,
You are right, the acking for the non-stateful bolts in a stateful topology is better handled by the bolts themselves.
I plan to submit a patch soon. You can track it here - https://issues.apache.org/jira/browse/STORM-1608
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Monday, March 7, 2016 at 4:37 PM
To: "user@storm.apache.org"
Subject: Re: Stateful bolts and acking
Hi Arun,
After reading your messages a bit more thoroughly (sorry :) ) I managed to get the basic case working. If I understand you correctly, one should inherit from BaseRichBolt, which normally requires manual acking, but *not* perform any manual acking. And one should *not* extends BaseBasicBolt which normally auto-acks, since this auto-acking will not work as intended with stateful topologies.
This would unfortunately mean that we cannot use bolts which were written for stateless topologies in stateful ones. Are there any plans of adapters or to change the interface so that they can interoperate?
Best regards
Alexander
On Fri, Mar 4, 2016 at 7:26 PM, Arun Mahadevan <ar...@apache.org> wrote:
Hi Alexander,
The simple topology like the one you have mentioned with a single spout and a stateful bolt with an empty execute works for me. I assume you are anchoring the tuple in the spout’s emit.
For a stateful topology, the acking doesn’t work if a bolt extends from BaseBasicBolt or if theres an “ack" call in the bolt’s execute.
You could open a JIRA (https://issues.apache.org/jira/browse/STORM) and share the sample topology code where you see this issue.
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, March 4, 2016 at 11:15 PM
To: "user@storm.apache.org"
Subject: Re: Stateful bolts and acking
Hi Arun,
Thank you for your reply. I have tried both manual and automatic acking, and neither works. Or rather, the acker is running but my spout is still receiving failure callbacks after the message timeout. I have a simple topology with a single spout reading from a file and a single empty bolt doing nothing. If I change from an empty stateless bolt to an empty stateful bolt the acking stops working.
Should I proceed with sending this in as a reproducing test case? Or is there anything else which I might have missed?
Regards,
Alexander
On Mar 4, 2016 5:30 PM, "Arun Mahadevan" <ar...@apache.org> wrote:
Hi Alexander,
For a stateful topology the anchoring and acking is automatically taken care of.
Can you check if any of your bolts inherit BaseBasicBolt or if you are manually acking. Your non-stateful bolts could inherit from BaseRichBolt instead.
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, March 4, 2016 at 8:04 PM
To: "user@storm.apache.org"
Subject: Stateful bolts and acking
Hello!
I'm on the 1.x-branch and trying to use the stateful bolts. But I cannot get it to work with acking. No matter what I try acking simply doesn't seem to have any effect. I see acking tasks suceeding in the logs (ack tuples forwarded to the acker, acker runs successfully), but still acking is not succesful and the tuples fail after the message timeout. I've not been successful in debugging the acker and the rotating map to figure out why acking is not completing. I was wondering if acking is supposed to work with stateful bolts, if so how and if there are any examples? Any ideas would be appreciated.
Best regards,
Alexander T
Re: Stateful bolts and acking
Posted by Alexander T <mi...@gmail.com>.
Hi Arun,
Great to hear about the patch you are working on. I have another question
if you have time: Right now I'm seeing acking a bit early in the chain,
specifically for a windowed bolt we have which extends BaseWindowedBolt.
The messages seem to be acked right before they reach this bolt, and if I
replace it with a simple stateful bolt, the acking starts working as
expected again. Is there support for non-stateful windowed bolts?
Best regards
Alexander
On Mon, Mar 7, 2016 at 12:16 PM, Arun Mahadevan <ar...@apache.org> wrote:
> Hi Alexander,
>
> You are right, the acking for the non-stateful bolts in a stateful
> topology is better handled by the bolts themselves.
>
> I plan to submit a patch soon. You can track it here -
> https://issues.apache.org/jira/browse/STORM-1608
>
> Thanks,
> Arun
>
> From: Alexander T
> Reply-To: "user@storm.apache.org"
> Date: Monday, March 7, 2016 at 4:37 PM
>
> To: "user@storm.apache.org"
> Subject: Re: Stateful bolts and acking
>
> Hi Arun,
>
> After reading your messages a bit more thoroughly (sorry :) ) I managed to
> get the basic case working. If I understand you correctly, one should
> inherit from BaseRichBolt, which normally requires manual acking, but *not*
> perform any manual acking. And one should *not* extends BaseBasicBolt which
> normally auto-acks, since this auto-acking will not work as intended with
> stateful topologies.
>
> This would unfortunately mean that we cannot use bolts which were written
> for stateless topologies in stateful ones. Are there any plans of adapters
> or to change the interface so that they can interoperate?
>
> Best regards
> Alexander
>
> On Fri, Mar 4, 2016 at 7:26 PM, Arun Mahadevan <ar...@apache.org> wrote:
>
>> Hi Alexander,
>>
>> The simple topology like the one you have mentioned with a single spout
>> and a stateful bolt with an empty execute works for me. I assume you are
>> anchoring the tuple in the spout’s emit.
>>
>> For a stateful topology, the acking doesn’t work if a bolt extends from
>> BaseBasicBolt or if theres an “ack" call in the bolt’s execute.
>>
>> You could open a JIRA (https://issues.apache.org/jira/browse/STORM) and
>> share the sample topology code where you see this issue.
>>
>> Thanks,
>> Arun
>>
>>
>> From: Alexander T
>> Reply-To: "user@storm.apache.org"
>> Date: Friday, March 4, 2016 at 11:15 PM
>> To: "user@storm.apache.org"
>> Subject: Re: Stateful bolts and acking
>>
>> Hi Arun,
>>
>> Thank you for your reply. I have tried both manual and automatic acking,
>> and neither works. Or rather, the acker is running but my spout is still
>> receiving failure callbacks after the message timeout. I have a simple
>> topology with a single spout reading from a file and a single empty bolt
>> doing nothing. If I change from an empty stateless bolt to an empty
>> stateful bolt the acking stops working.
>>
>> Should I proceed with sending this in as a reproducing test case? Or is
>> there anything else which I might have missed?
>>
>> Regards,
>> Alexander
>> On Mar 4, 2016 5:30 PM, "Arun Mahadevan" <ar...@apache.org> wrote:
>>
>>> Hi Alexander,
>>>
>>> For a stateful topology the anchoring and acking is automatically taken
>>> care of.
>>>
>>> Can you check if any of your bolts inherit BaseBasicBolt or if you are
>>> manually acking. Your non-stateful bolts could inherit from BaseRichBolt
>>> instead.
>>>
>>> Thanks,
>>> Arun
>>>
>>>
>>> From: Alexander T
>>> Reply-To: "user@storm.apache.org"
>>> Date: Friday, March 4, 2016 at 8:04 PM
>>> To: "user@storm.apache.org"
>>> Subject: Stateful bolts and acking
>>>
>>> Hello!
>>>
>>> I'm on the 1.x-branch and trying to use the stateful bolts. But I cannot
>>> get it to work with acking. No matter what I try acking simply doesn't seem
>>> to have any effect. I see acking tasks suceeding in the logs (ack tuples
>>> forwarded to the acker, acker runs successfully), but still acking is not
>>> succesful and the tuples fail after the message timeout. I've not been
>>> successful in debugging the acker and the rotating map to figure out why
>>> acking is not completing. I was wondering if acking is supposed to work
>>> with stateful bolts, if so how and if there are any examples? Any ideas
>>> would be appreciated.
>>>
>>> Best regards,
>>>
>>> Alexander T
>>>
>>
>
Re: Stateful bolts and acking
Posted by Arun Mahadevan <ar...@apache.org>.
Hi Alexander,
You are right, the acking for the non-stateful bolts in a stateful topology is better handled by the bolts themselves.
I plan to submit a patch soon. You can track it here - https://issues.apache.org/jira/browse/STORM-1608
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Monday, March 7, 2016 at 4:37 PM
To: "user@storm.apache.org"
Subject: Re: Stateful bolts and acking
Hi Arun,
After reading your messages a bit more thoroughly (sorry :) ) I managed to get the basic case working. If I understand you correctly, one should inherit from BaseRichBolt, which normally requires manual acking, but *not* perform any manual acking. And one should *not* extends BaseBasicBolt which normally auto-acks, since this auto-acking will not work as intended with stateful topologies.
This would unfortunately mean that we cannot use bolts which were written for stateless topologies in stateful ones. Are there any plans of adapters or to change the interface so that they can interoperate?
Best regards
Alexander
On Fri, Mar 4, 2016 at 7:26 PM, Arun Mahadevan <ar...@apache.org> wrote:
Hi Alexander,
The simple topology like the one you have mentioned with a single spout and a stateful bolt with an empty execute works for me. I assume you are anchoring the tuple in the spout’s emit.
For a stateful topology, the acking doesn’t work if a bolt extends from BaseBasicBolt or if theres an “ack" call in the bolt’s execute.
You could open a JIRA (https://issues.apache.org/jira/browse/STORM) and share the sample topology code where you see this issue.
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, March 4, 2016 at 11:15 PM
To: "user@storm.apache.org"
Subject: Re: Stateful bolts and acking
Hi Arun,
Thank you for your reply. I have tried both manual and automatic acking, and neither works. Or rather, the acker is running but my spout is still receiving failure callbacks after the message timeout. I have a simple topology with a single spout reading from a file and a single empty bolt doing nothing. If I change from an empty stateless bolt to an empty stateful bolt the acking stops working.
Should I proceed with sending this in as a reproducing test case? Or is there anything else which I might have missed?
Regards,
Alexander
On Mar 4, 2016 5:30 PM, "Arun Mahadevan" <ar...@apache.org> wrote:
Hi Alexander,
For a stateful topology the anchoring and acking is automatically taken care of.
Can you check if any of your bolts inherit BaseBasicBolt or if you are manually acking. Your non-stateful bolts could inherit from BaseRichBolt instead.
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, March 4, 2016 at 8:04 PM
To: "user@storm.apache.org"
Subject: Stateful bolts and acking
Hello!
I'm on the 1.x-branch and trying to use the stateful bolts. But I cannot get it to work with acking. No matter what I try acking simply doesn't seem to have any effect. I see acking tasks suceeding in the logs (ack tuples forwarded to the acker, acker runs successfully), but still acking is not succesful and the tuples fail after the message timeout. I've not been successful in debugging the acker and the rotating map to figure out why acking is not completing. I was wondering if acking is supposed to work with stateful bolts, if so how and if there are any examples? Any ideas would be appreciated.
Best regards,
Alexander T
Re: Stateful bolts and acking
Posted by Alexander T <mi...@gmail.com>.
Hi Arun,
After reading your messages a bit more thoroughly (sorry :) ) I managed to
get the basic case working. If I understand you correctly, one should
inherit from BaseRichBolt, which normally requires manual acking, but *not*
perform any manual acking. And one should *not* extends BaseBasicBolt which
normally auto-acks, since this auto-acking will not work as intended with
stateful topologies.
This would unfortunately mean that we cannot use bolts which were written
for stateless topologies in stateful ones. Are there any plans of adapters
or to change the interface so that they can interoperate?
Best regards
Alexander
On Fri, Mar 4, 2016 at 7:26 PM, Arun Mahadevan <ar...@apache.org> wrote:
> Hi Alexander,
>
> The simple topology like the one you have mentioned with a single spout
> and a stateful bolt with an empty execute works for me. I assume you are
> anchoring the tuple in the spout’s emit.
>
> For a stateful topology, the acking doesn’t work if a bolt extends from
> BaseBasicBolt or if theres an “ack" call in the bolt’s execute.
>
> You could open a JIRA (https://issues.apache.org/jira/browse/STORM) and
> share the sample topology code where you see this issue.
>
> Thanks,
> Arun
>
>
> From: Alexander T
> Reply-To: "user@storm.apache.org"
> Date: Friday, March 4, 2016 at 11:15 PM
> To: "user@storm.apache.org"
> Subject: Re: Stateful bolts and acking
>
> Hi Arun,
>
> Thank you for your reply. I have tried both manual and automatic acking,
> and neither works. Or rather, the acker is running but my spout is still
> receiving failure callbacks after the message timeout. I have a simple
> topology with a single spout reading from a file and a single empty bolt
> doing nothing. If I change from an empty stateless bolt to an empty
> stateful bolt the acking stops working.
>
> Should I proceed with sending this in as a reproducing test case? Or is
> there anything else which I might have missed?
>
> Regards,
> Alexander
> On Mar 4, 2016 5:30 PM, "Arun Mahadevan" <ar...@apache.org> wrote:
>
>> Hi Alexander,
>>
>> For a stateful topology the anchoring and acking is automatically taken
>> care of.
>>
>> Can you check if any of your bolts inherit BaseBasicBolt or if you are
>> manually acking. Your non-stateful bolts could inherit from BaseRichBolt
>> instead.
>>
>> Thanks,
>> Arun
>>
>>
>> From: Alexander T
>> Reply-To: "user@storm.apache.org"
>> Date: Friday, March 4, 2016 at 8:04 PM
>> To: "user@storm.apache.org"
>> Subject: Stateful bolts and acking
>>
>> Hello!
>>
>> I'm on the 1.x-branch and trying to use the stateful bolts. But I cannot
>> get it to work with acking. No matter what I try acking simply doesn't seem
>> to have any effect. I see acking tasks suceeding in the logs (ack tuples
>> forwarded to the acker, acker runs successfully), but still acking is not
>> succesful and the tuples fail after the message timeout. I've not been
>> successful in debugging the acker and the rotating map to figure out why
>> acking is not completing. I was wondering if acking is supposed to work
>> with stateful bolts, if so how and if there are any examples? Any ideas
>> would be appreciated.
>>
>> Best regards,
>>
>> Alexander T
>>
>
Re: Stateful bolts and acking
Posted by Arun Mahadevan <ar...@apache.org>.
Hi Alexander,
The simple topology like the one you have mentioned with a single spout and a stateful bolt with an empty execute works for me. I assume you are anchoring the tuple in the spout’s emit.
For a stateful topology, the acking doesn’t work if a bolt extends from BaseBasicBolt or if theres an “ack" call in the bolt’s execute.
You could open a JIRA (https://issues.apache.org/jira/browse/STORM) and share the sample topology code where you see this issue.
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, March 4, 2016 at 11:15 PM
To: "user@storm.apache.org"
Subject: Re: Stateful bolts and acking
Hi Arun,
Thank you for your reply. I have tried both manual and automatic acking, and neither works. Or rather, the acker is running but my spout is still receiving failure callbacks after the message timeout. I have a simple topology with a single spout reading from a file and a single empty bolt doing nothing. If I change from an empty stateless bolt to an empty stateful bolt the acking stops working.
Should I proceed with sending this in as a reproducing test case? Or is there anything else which I might have missed?
Regards,
Alexander
On Mar 4, 2016 5:30 PM, "Arun Mahadevan" <ar...@apache.org> wrote:
Hi Alexander,
For a stateful topology the anchoring and acking is automatically taken care of.
Can you check if any of your bolts inherit BaseBasicBolt or if you are manually acking. Your non-stateful bolts could inherit from BaseRichBolt instead.
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, March 4, 2016 at 8:04 PM
To: "user@storm.apache.org"
Subject: Stateful bolts and acking
Hello!
I'm on the 1.x-branch and trying to use the stateful bolts. But I cannot get it to work with acking. No matter what I try acking simply doesn't seem to have any effect. I see acking tasks suceeding in the logs (ack tuples forwarded to the acker, acker runs successfully), but still acking is not succesful and the tuples fail after the message timeout. I've not been successful in debugging the acker and the rotating map to figure out why acking is not completing. I was wondering if acking is supposed to work with stateful bolts, if so how and if there are any examples? Any ideas would be appreciated.
Best regards,
Alexander T
Re: Stateful bolts and acking
Posted by Alexander T <mi...@gmail.com>.
Hi Arun,
Thank you for your reply. I have tried both manual and automatic acking,
and neither works. Or rather, the acker is running but my spout is still
receiving failure callbacks after the message timeout. I have a simple
topology with a single spout reading from a file and a single empty bolt
doing nothing. If I change from an empty stateless bolt to an empty
stateful bolt the acking stops working.
Should I proceed with sending this in as a reproducing test case? Or is
there anything else which I might have missed?
Regards,
Alexander
On Mar 4, 2016 5:30 PM, "Arun Mahadevan" <ar...@apache.org> wrote:
> Hi Alexander,
>
> For a stateful topology the anchoring and acking is automatically taken
> care of.
>
> Can you check if any of your bolts inherit BaseBasicBolt or if you are
> manually acking. Your non-stateful bolts could inherit from BaseRichBolt
> instead.
>
> Thanks,
> Arun
>
>
> From: Alexander T
> Reply-To: "user@storm.apache.org"
> Date: Friday, March 4, 2016 at 8:04 PM
> To: "user@storm.apache.org"
> Subject: Stateful bolts and acking
>
> Hello!
>
> I'm on the 1.x-branch and trying to use the stateful bolts. But I cannot
> get it to work with acking. No matter what I try acking simply doesn't seem
> to have any effect. I see acking tasks suceeding in the logs (ack tuples
> forwarded to the acker, acker runs successfully), but still acking is not
> succesful and the tuples fail after the message timeout. I've not been
> successful in debugging the acker and the rotating map to figure out why
> acking is not completing. I was wondering if acking is supposed to work
> with stateful bolts, if so how and if there are any examples? Any ideas
> would be appreciated.
>
> Best regards,
>
> Alexander T
>
Re: Stateful bolts and acking
Posted by Arun Mahadevan <ar...@apache.org>.
Hi Alexander,
For a stateful topology the anchoring and acking is automatically taken care of.
Can you check if any of your bolts inherit BaseBasicBolt or if you are manually acking. Your non-stateful bolts could inherit from BaseRichBolt instead.
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, March 4, 2016 at 8:04 PM
To: "user@storm.apache.org"
Subject: Stateful bolts and acking
Hello!
I'm on the 1.x-branch and trying to use the stateful bolts. But I cannot get it to work with acking. No matter what I try acking simply doesn't seem to have any effect. I see acking tasks suceeding in the logs (ack tuples forwarded to the acker, acker runs successfully), but still acking is not succesful and the tuples fail after the message timeout. I've not been successful in debugging the acker and the rotating map to figure out why acking is not completing. I was wondering if acking is supposed to work with stateful bolts, if so how and if there are any examples? Any ideas would be appreciated.
Best regards,
Alexander T