How to recover from a Spout excpetion?

[Alexis Sukrieh <su...@gmail.com>]

Hi,

I have built a topology with a spout that connects to Twitter's streaming
API (with twitter4j), I have also 4 bolts that count words inside.

This is mainly a proof of concept for me, in order to get up to speed with
Storm.

So far, it's working fine... until my Spout gets an exception from
twitter4j:

java.lang.RuntimeException: java.lang.RuntimeException:
java.io.NotSerializableException: twitter4j.ScopesImpl

When this happens, my topology is somehow broken: the Redis database
that gets updated by the last bolt is not updated anymore.

All I can do is killing the topology and submit it again. That's a bit
annoying as I'd like my topology to never stops running.

I'd like to understand how to catch this exception, and make sure my
topology is protected against invalid tuples.

In the Spout class, I really don't see how I could catch the exception.

Any help is more than welcome.


Thanks.


-- 
Alexis Sukrieh

Re: How to recover from a Spout excpetion?

[Alexis Sukrieh <su...@gmail.com>]

Thanks Cody, patching the ScopesImpl class is indeed the root solution,
I'll see if I can do that.

Meanwhile, is there a way for me to automatically kill/resubmit my topology
when I encounter this error?

​I realized that exiting when an exception is caught is not enough, the
workers are restarted but the topology remains in a bogus state.
​
​Thanks.​


2014-05-22 23:32 GMT+02:00 Cody A. Ray <co...@gmail.com>:

> I think the problem is that the code bundled with your topology is
> serialized by the nimbus and transferred to the supervisors, so all the
> code you use needs to be serializable. Something in twitter4j must be a
> using a non-serializable class (the ScopesImpl class).
>
> I don't have a suggestion other than patch/contribute twitter4j or look at
> a different twitter library... But I think that's the root cause.
>
> Someone else may have a better suggestion.
>
> -Cody
> On May 22, 2014 1:00 PM, "Alexis Sukrieh" <su...@gmail.com> wrote:
>
>> Hi,
>>
>> I have built a topology with a spout that connects to Twitter's streaming
>> API (with twitter4j), I have also 4 bolts that count words inside.
>>
>> This is mainly a proof of concept for me, in order to get up to speed
>> with Storm.
>>
>> So far, it's working fine... until my Spout gets an exception from
>> twitter4j:
>>
>> java.lang.RuntimeException: java.lang.RuntimeException: java.io.NotSerializableException: twitter4j.ScopesImpl
>>
>> When this happens, my topology is somehow broken: the Redis database that gets updated by the last bolt is not updated anymore.
>>
>>
>> All I can do is killing the topology and submit it again. That's a bit annoying as I'd like my topology to never stops running.
>>
>> I'd like to understand how to catch this exception, and make sure my topology is protected against invalid tuples.
>>
>>
>> In the Spout class, I really don't see how I could catch the exception.
>>
>> Any help is more than welcome.
>>
>>
>> Thanks.
>>
>>
>> --
>> Alexis Sukrieh
>>
>>


-- 
Alexis Sukrieh

http://twitter.com/sukria - http://blog.sukria.net

"People get annoyed when you try to debug them." -- Larry Wall

Re: How to recover from a Spout excpetion?

["Cody A. Ray" <co...@gmail.com>]

I think the problem is that the code bundled with your topology is
serialized by the nimbus and transferred to the supervisors, so all the
code you use needs to be serializable. Something in twitter4j must be a
using a non-serializable class (the ScopesImpl class).

I don't have a suggestion other than patch/contribute twitter4j or look at
a different twitter library... But I think that's the root cause.

Someone else may have a better suggestion.

-Cody
On May 22, 2014 1:00 PM, "Alexis Sukrieh" <su...@gmail.com> wrote:

> Hi,
>
> I have built a topology with a spout that connects to Twitter's streaming
> API (with twitter4j), I have also 4 bolts that count words inside.
>
> This is mainly a proof of concept for me, in order to get up to speed with
> Storm.
>
> So far, it's working fine... until my Spout gets an exception from
> twitter4j:
>
> java.lang.RuntimeException: java.lang.RuntimeException: java.io.NotSerializableException: twitter4j.ScopesImpl
>
> When this happens, my topology is somehow broken: the Redis database that gets updated by the last bolt is not updated anymore.
>
> All I can do is killing the topology and submit it again. That's a bit annoying as I'd like my topology to never stops running.
>
> I'd like to understand how to catch this exception, and make sure my topology is protected against invalid tuples.
>
> In the Spout class, I really don't see how I could catch the exception.
>
> Any help is more than welcome.
>
>
> Thanks.
>
>
> --
> Alexis Sukrieh
>
>