You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Spico Florin <sp...@gmail.com> on 2016/04/15 10:33:18 UTC
initState method not invoked in Storm 1.0
Hello!
I'm running a topology in LocalCluster that has a stasteful Bolt. Wile
debugging, I have observed that the initState method is not invoked at all.
The documentation said:
"The initState method is invoked by the framework during the bolt
initialization with the previously saved state of the bolt. This is invoked
after prepare but before the bolt starts processing any tuples".
Due to this, the state field remains null and I get NPE when I populate it
with state .put
Any idea why the initState is not invoked?
Regards,
Florin
Here is my code:
public class TimeSeriesStatefulBolt extends
BaseStatefulBolt<KeyValueState<Long, Map<String, Float>>> {
private KeyValueState<Long, Map<String, Float>> state;
@Override
public void initState(KeyValueState<Long, Map<String, Float>> state) {
this.state = state;
}
Re: initState method not invoked in Storm 1.0
Posted by Arun Mahadevan <ar...@apache.org>.
Yes the static dispatch of overloads can cause issues here. Plan to address in https://issues.apache.org/jira/browse/STORM-1714. Thanks for bringing this up.
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, April 15, 2016 at 3:02 PM
To: "user@storm.apache.org"
Subject: Re: initState method not invoked in Storm 1.0
Hi Arun,
Yes, that is one way of getting it wrong. Another way is assigning your bolt to a variable of a stateless type before doing setBolt. Since Java lacks dynamic dispatch, the stateless overload will be chosen in that case.
Regards,
Alex
On Apr 15, 2016 11:16 AM, "Arun Mahadevan" <ar...@apache.org> wrote:
Ah, I see what you mean. The “setBolt” method without parallelism hint is not overloaded for stateful bolts so if parallelism hint is not specified it ends up as being normal bolt. Will raise a JIRA for fixing this.
Spico,
For now, can you provide parallelism hint while you add stateful bolt to the topology ?
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, April 15, 2016 at 2:38 PM
To: "user@storm.apache.org"
Subject: Re: initState method not invoked in Storm 1.0
Hi Arun,
I meant that it's very easy to use the wrong setBolt method overload by mistake, since stateful bolts are supertypes of stateless ones.
Regards
Alex
On Apr 15, 2016 10:54 AM, "Arun Mahadevan" <ar...@apache.org> wrote:
Its the same method (builder.setBolt) that adds stateful bolts to a topology. Heres an example - https://github.com/apache/storm/blob/master/examples/storm-starter/src/jvm/storm/starter/StatefulTopology.java
Spico,
Do you see any errors in the logs ? You might want to turn on debug logs and see whats happening. Can you also try running the StatefulTopology in the storm-starter and check if you see the same behavior ?
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, April 15, 2016 at 2:06 PM
To: "user@storm.apache.org"
Subject: Re: initState method not invoked in Storm 1.0
Hi Spico,
Are you adding your bolt to the topology with the special methods for stateful bolts? It's quite easy to use the regular addBolt method and it will in that case be treated as a stateless one.
Cheers
Alex
On Apr 15, 2016 10:33 AM, "Spico Florin" <sp...@gmail.com> wrote:
Hello!
I'm running a topology in LocalCluster that has a stasteful Bolt. Wile debugging, I have observed that the initState method is not invoked at all.
The documentation said:
"The initState method is invoked by the framework during the bolt initialization with the previously saved state of the bolt. This is invoked after prepare but before the bolt starts processing any tuples".
Due to this, the state field remains null and I get NPE when I populate it with state .put
Any idea why the initState is not invoked?
Regards,
Florin
Here is my code:
public class TimeSeriesStatefulBolt extends
BaseStatefulBolt<KeyValueState<Long, Map<String, Float>>> {
private KeyValueState<Long, Map<String, Float>> state;
@Override
public void initState(KeyValueState<Long, Map<String, Float>> state) {
this.state = state;
}
Re: initState method not invoked in Storm 1.0
Posted by Alexander T <mi...@gmail.com>.
Hi Arun,
Yes, that is one way of getting it wrong. Another way is assigning your
bolt to a variable of a stateless type before doing setBolt. Since Java
lacks dynamic dispatch, the stateless overload will be chosen in that case.
Regards,
Alex
On Apr 15, 2016 11:16 AM, "Arun Mahadevan" <ar...@apache.org> wrote:
> Ah, I see what you mean. The “setBolt” method without parallelism hint is
> not overloaded for stateful bolts so if parallelism hint is not specified
> it ends up as being normal bolt. Will raise a JIRA for fixing this.
>
> Spico,
>
> For now, can you provide parallelism hint while you add stateful bolt to
> the topology ?
>
> Thanks,
> Arun
>
> From: Alexander T
> Reply-To: "user@storm.apache.org"
> Date: Friday, April 15, 2016 at 2:38 PM
> To: "user@storm.apache.org"
> Subject: Re: initState method not invoked in Storm 1.0
>
> Hi Arun,
>
> I meant that it's very easy to use the wrong setBolt method overload by
> mistake, since stateful bolts are supertypes of stateless ones.
>
> Regards
> Alex
> On Apr 15, 2016 10:54 AM, "Arun Mahadevan" <ar...@apache.org> wrote:
>
> Its the same method (builder.setBolt) that adds stateful bolts to a
> topology. Heres an example -
> https://github.com/apache/storm/blob/master/examples/storm-starter/src/jvm/storm/starter/StatefulTopology.java
>
> Spico,
>
> Do you see any errors in the logs ? You might want to turn on debug logs
> and see whats happening. Can you also try running the StatefulTopology in
> the storm-starter and check if you see the same behavior ?
>
> Thanks,
> Arun
>
> From: Alexander T
> Reply-To: "user@storm.apache.org"
> Date: Friday, April 15, 2016 at 2:06 PM
> To: "user@storm.apache.org"
> Subject: Re: initState method not invoked in Storm 1.0
>
> Hi Spico,
>
> Are you adding your bolt to the topology with the special methods for
> stateful bolts? It's quite easy to use the regular addBolt method and it
> will in that case be treated as a stateless one.
>
> Cheers
> Alex
> On Apr 15, 2016 10:33 AM, "Spico Florin" <sp...@gmail.com> wrote:
>
>> Hello!
>> I'm running a topology in LocalCluster that has a stasteful Bolt. Wile
>> debugging, I have observed that the initState method is not invoked at all.
>> The documentation said:
>> "The initState method is invoked by the framework during the bolt
>> initialization with the previously saved state of the bolt. This is invoked
>> after prepare but before the bolt starts processing any tuples".
>>
>> Due to this, the state field remains null and I get NPE when I populate
>> it with state .put
>> Any idea why the initState is not invoked?
>> Regards,
>> Florin
>>
>> Here is my code:
>>
>> public class TimeSeriesStatefulBolt extends
>> BaseStatefulBolt<KeyValueState<Long, Map<String, Float>>> {
>>
>> private KeyValueState<Long, Map<String, Float>> state;
>> @Override
>> public void initState(KeyValueState<Long, Map<String, Float>> state) {
>> this.state = state;
>> }
>>
>
Re: initState method not invoked in Storm 1.0
Posted by Arun Mahadevan <ar...@apache.org>.
Yes, In the stateful bolts you need to anchor the tuples while emitting and ack them.
If you have any non-stateful bolts in the same topology they should also do the anchoring/acking (or extend BaseBasicBolt).
From: Spico Florin
Reply-To: "user@storm.apache.org"
Date: Friday, April 15, 2016 at 2:52 PM
To: "user@storm.apache.org"
Subject: Re: initState method not invoked in Storm 1.0
Hi!
Thank you for your answers. Yes I will provide the hint parallelism. Regarding, the constraint that I'm facing should I emit from both spout and stateful bolt anchored tuples (that way the the exception was complaining)?
I look forward for your answers,
Florin
On Fri, Apr 15, 2016 at 12:16 PM, Arun Mahadevan <ar...@apache.org> wrote:
Ah, I see what you mean. The “setBolt” method without parallelism hint is not overloaded for stateful bolts so if parallelism hint is not specified it ends up as being normal bolt. Will raise a JIRA for fixing this.
Spico,
For now, can you provide parallelism hint while you add stateful bolt to the topology ?
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, April 15, 2016 at 2:38 PM
To: "user@storm.apache.org"
Subject: Re: initState method not invoked in Storm 1.0
Hi Arun,
I meant that it's very easy to use the wrong setBolt method overload by mistake, since stateful bolts are supertypes of stateless ones.
Regards
Alex
On Apr 15, 2016 10:54 AM, "Arun Mahadevan" <ar...@apache.org> wrote:
Its the same method (builder.setBolt) that adds stateful bolts to a topology. Heres an example - https://github.com/apache/storm/blob/master/examples/storm-starter/src/jvm/storm/starter/StatefulTopology.java
Spico,
Do you see any errors in the logs ? You might want to turn on debug logs and see whats happening. Can you also try running the StatefulTopology in the storm-starter and check if you see the same behavior ?
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, April 15, 2016 at 2:06 PM
To: "user@storm.apache.org"
Subject: Re: initState method not invoked in Storm 1.0
Hi Spico,
Are you adding your bolt to the topology with the special methods for stateful bolts? It's quite easy to use the regular addBolt method and it will in that case be treated as a stateless one.
Cheers
Alex
On Apr 15, 2016 10:33 AM, "Spico Florin" <sp...@gmail.com> wrote:
Hello!
I'm running a topology in LocalCluster that has a stasteful Bolt. Wile debugging, I have observed that the initState method is not invoked at all.
The documentation said:
"The initState method is invoked by the framework during the bolt initialization with the previously saved state of the bolt. This is invoked after prepare but before the bolt starts processing any tuples".
Due to this, the state field remains null and I get NPE when I populate it with state .put
Any idea why the initState is not invoked?
Regards,
Florin
Here is my code:
public class TimeSeriesStatefulBolt extends
BaseStatefulBolt<KeyValueState<Long, Map<String, Float>>> {
private KeyValueState<Long, Map<String, Float>> state;
@Override
public void initState(KeyValueState<Long, Map<String, Float>> state) {
this.state = state;
}
Re: initState method not invoked in Storm 1.0
Posted by Spico Florin <sp...@gmail.com>.
Hi!
Thank you for your answers. Yes I will provide the hint parallelism.
Regarding, the constraint that I'm facing should I emit from both spout and
stateful bolt anchored tuples (that way the the exception was complaining)?
I look forward for your answers,
Florin
On Fri, Apr 15, 2016 at 12:16 PM, Arun Mahadevan <ar...@apache.org> wrote:
> Ah, I see what you mean. The “setBolt” method without parallelism hint is
> not overloaded for stateful bolts so if parallelism hint is not specified
> it ends up as being normal bolt. Will raise a JIRA for fixing this.
>
> Spico,
>
> For now, can you provide parallelism hint while you add stateful bolt to
> the topology ?
>
> Thanks,
> Arun
>
> From: Alexander T
> Reply-To: "user@storm.apache.org"
> Date: Friday, April 15, 2016 at 2:38 PM
>
> To: "user@storm.apache.org"
> Subject: Re: initState method not invoked in Storm 1.0
>
> Hi Arun,
>
> I meant that it's very easy to use the wrong setBolt method overload by
> mistake, since stateful bolts are supertypes of stateless ones.
>
> Regards
> Alex
> On Apr 15, 2016 10:54 AM, "Arun Mahadevan" <ar...@apache.org> wrote:
>
> Its the same method (builder.setBolt) that adds stateful bolts to a
> topology. Heres an example -
> https://github.com/apache/storm/blob/master/examples/storm-starter/src/jvm/storm/starter/StatefulTopology.java
>
> Spico,
>
> Do you see any errors in the logs ? You might want to turn on debug logs
> and see whats happening. Can you also try running the StatefulTopology in
> the storm-starter and check if you see the same behavior ?
>
> Thanks,
> Arun
>
> From: Alexander T
> Reply-To: "user@storm.apache.org"
> Date: Friday, April 15, 2016 at 2:06 PM
> To: "user@storm.apache.org"
> Subject: Re: initState method not invoked in Storm 1.0
>
> Hi Spico,
>
> Are you adding your bolt to the topology with the special methods for
> stateful bolts? It's quite easy to use the regular addBolt method and it
> will in that case be treated as a stateless one.
>
> Cheers
> Alex
> On Apr 15, 2016 10:33 AM, "Spico Florin" <sp...@gmail.com> wrote:
>
>> Hello!
>> I'm running a topology in LocalCluster that has a stasteful Bolt. Wile
>> debugging, I have observed that the initState method is not invoked at all.
>> The documentation said:
>> "The initState method is invoked by the framework during the bolt
>> initialization with the previously saved state of the bolt. This is invoked
>> after prepare but before the bolt starts processing any tuples".
>>
>> Due to this, the state field remains null and I get NPE when I populate
>> it with state .put
>> Any idea why the initState is not invoked?
>> Regards,
>> Florin
>>
>> Here is my code:
>>
>> public class TimeSeriesStatefulBolt extends
>> BaseStatefulBolt<KeyValueState<Long, Map<String, Float>>> {
>>
>> private KeyValueState<Long, Map<String, Float>> state;
>> @Override
>> public void initState(KeyValueState<Long, Map<String, Float>> state) {
>> this.state = state;
>> }
>>
>
Re: initState method not invoked in Storm 1.0
Posted by Arun Mahadevan <ar...@apache.org>.
Ah, I see what you mean. The “setBolt” method without parallelism hint is not overloaded for stateful bolts so if parallelism hint is not specified it ends up as being normal bolt. Will raise a JIRA for fixing this.
Spico,
For now, can you provide parallelism hint while you add stateful bolt to the topology ?
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, April 15, 2016 at 2:38 PM
To: "user@storm.apache.org"
Subject: Re: initState method not invoked in Storm 1.0
Hi Arun,
I meant that it's very easy to use the wrong setBolt method overload by mistake, since stateful bolts are supertypes of stateless ones.
Regards
Alex
On Apr 15, 2016 10:54 AM, "Arun Mahadevan" <ar...@apache.org> wrote:
Its the same method (builder.setBolt) that adds stateful bolts to a topology. Heres an example - https://github.com/apache/storm/blob/master/examples/storm-starter/src/jvm/storm/starter/StatefulTopology.java
Spico,
Do you see any errors in the logs ? You might want to turn on debug logs and see whats happening. Can you also try running the StatefulTopology in the storm-starter and check if you see the same behavior ?
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, April 15, 2016 at 2:06 PM
To: "user@storm.apache.org"
Subject: Re: initState method not invoked in Storm 1.0
Hi Spico,
Are you adding your bolt to the topology with the special methods for stateful bolts? It's quite easy to use the regular addBolt method and it will in that case be treated as a stateless one.
Cheers
Alex
On Apr 15, 2016 10:33 AM, "Spico Florin" <sp...@gmail.com> wrote:
Hello!
I'm running a topology in LocalCluster that has a stasteful Bolt. Wile debugging, I have observed that the initState method is not invoked at all.
The documentation said:
"The initState method is invoked by the framework during the bolt initialization with the previously saved state of the bolt. This is invoked after prepare but before the bolt starts processing any tuples".
Due to this, the state field remains null and I get NPE when I populate it with state .put
Any idea why the initState is not invoked?
Regards,
Florin
Here is my code:
public class TimeSeriesStatefulBolt extends
BaseStatefulBolt<KeyValueState<Long, Map<String, Float>>> {
private KeyValueState<Long, Map<String, Float>> state;
@Override
public void initState(KeyValueState<Long, Map<String, Float>> state) {
this.state = state;
}
Re: initState method not invoked in Storm 1.0
Posted by Alexander T <mi...@gmail.com>.
Hi Arun,
I meant that it's very easy to use the wrong setBolt method overload by
mistake, since stateful bolts are supertypes of stateless ones.
Regards
Alex
On Apr 15, 2016 10:54 AM, "Arun Mahadevan" <ar...@apache.org> wrote:
Its the same method (builder.setBolt) that adds stateful bolts to a
topology. Heres an example -
https://github.com/apache/storm/blob/master/examples/storm-starter/src/jvm/storm/starter/StatefulTopology.java
Spico,
Do you see any errors in the logs ? You might want to turn on debug logs
and see whats happening. Can you also try running the StatefulTopology in
the storm-starter and check if you see the same behavior ?
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, April 15, 2016 at 2:06 PM
To: "user@storm.apache.org"
Subject: Re: initState method not invoked in Storm 1.0
Hi Spico,
Are you adding your bolt to the topology with the special methods for
stateful bolts? It's quite easy to use the regular addBolt method and it
will in that case be treated as a stateless one.
Cheers
Alex
On Apr 15, 2016 10:33 AM, "Spico Florin" <sp...@gmail.com> wrote:
> Hello!
> I'm running a topology in LocalCluster that has a stasteful Bolt. Wile
> debugging, I have observed that the initState method is not invoked at all.
> The documentation said:
> "The initState method is invoked by the framework during the bolt
> initialization with the previously saved state of the bolt. This is invoked
> after prepare but before the bolt starts processing any tuples".
>
> Due to this, the state field remains null and I get NPE when I populate it
> with state .put
> Any idea why the initState is not invoked?
> Regards,
> Florin
>
> Here is my code:
>
> public class TimeSeriesStatefulBolt extends
> BaseStatefulBolt<KeyValueState<Long, Map<String, Float>>> {
>
> private KeyValueState<Long, Map<String, Float>> state;
> @Override
> public void initState(KeyValueState<Long, Map<String, Float>> state) {
> this.state = state;
> }
>
Re: initState method not invoked in Storm 1.0
Posted by Arun Mahadevan <ar...@apache.org>.
Its the same method (builder.setBolt) that adds stateful bolts to a topology. Heres an example - https://github.com/apache/storm/blob/master/examples/storm-starter/src/jvm/storm/starter/StatefulTopology.java
Spico,
Do you see any errors in the logs ? You might want to turn on debug logs and see whats happening. Can you also try running the StatefulTopology in the storm-starter and check if you see the same behavior ?
Thanks,
Arun
From: Alexander T
Reply-To: "user@storm.apache.org"
Date: Friday, April 15, 2016 at 2:06 PM
To: "user@storm.apache.org"
Subject: Re: initState method not invoked in Storm 1.0
Hi Spico,
Are you adding your bolt to the topology with the special methods for stateful bolts? It's quite easy to use the regular addBolt method and it will in that case be treated as a stateless one.
Cheers
Alex
On Apr 15, 2016 10:33 AM, "Spico Florin" <sp...@gmail.com> wrote:
Hello!
I'm running a topology in LocalCluster that has a stasteful Bolt. Wile debugging, I have observed that the initState method is not invoked at all.
The documentation said:
"The initState method is invoked by the framework during the bolt initialization with the previously saved state of the bolt. This is invoked after prepare but before the bolt starts processing any tuples".
Due to this, the state field remains null and I get NPE when I populate it with state .put
Any idea why the initState is not invoked?
Regards,
Florin
Here is my code:
public class TimeSeriesStatefulBolt extends
BaseStatefulBolt<KeyValueState<Long, Map<String, Float>>> {
private KeyValueState<Long, Map<String, Float>> state;
@Override
public void initState(KeyValueState<Long, Map<String, Float>> state) {
this.state = state;
}
Re: initState method not invoked in Storm 1.0
Posted by Spico Florin <sp...@gmail.com>.
Hi, Alex!
There is no special method addBolt in the API TopologyBuilder for . I
have used the topologyBuilder.setBolt("myId", new TimeSeriesStatefulBolt())
without parallelism_hint declared and the API dismiss the statefulness.
I did NOT get any errors in the Logs.
What I have found is the topologyBuilder.setBolt(String id, IStatefulBolt
bolt, Number parallelism_hint) is considering the statefulness of the bolt,
but now I'm facing a different issue that seems to be related with some
constraints of the stareful bolts:
java.lang.RuntimeException: java.lang.UnsupportedOperationException: Bolts
in a stateful topology must emit anchored tuples.
Thank you for your support.
Florin
On Fri, Apr 15, 2016 at 11:36 AM, Alexander T <mi...@gmail.com>
wrote:
> Hi Spico,
>
> Are you adding your bolt to the topology with the special methods for
> stateful bolts? It's quite easy to use the regular addBolt method and it
> will in that case be treated as a stateless one.
>
> Cheers
> Alex
> On Apr 15, 2016 10:33 AM, "Spico Florin" <sp...@gmail.com> wrote:
>
>> Hello!
>> I'm running a topology in LocalCluster that has a stasteful Bolt. Wile
>> debugging, I have observed that the initState method is not invoked at all.
>> The documentation said:
>> "The initState method is invoked by the framework during the bolt
>> initialization with the previously saved state of the bolt. This is invoked
>> after prepare but before the bolt starts processing any tuples".
>>
>> Due to this, the state field remains null and I get NPE when I populate
>> it with state .put
>> Any idea why the initState is not invoked?
>> Regards,
>> Florin
>>
>> Here is my code:
>>
>> public class TimeSeriesStatefulBolt extends
>> BaseStatefulBolt<KeyValueState<Long, Map<String, Float>>> {
>>
>> private KeyValueState<Long, Map<String, Float>> state;
>> @Override
>> public void initState(KeyValueState<Long, Map<String, Float>> state) {
>> this.state = state;
>> }
>>
>
Re: initState method not invoked in Storm 1.0
Posted by Alexander T <mi...@gmail.com>.
Hi Spico,
Are you adding your bolt to the topology with the special methods for
stateful bolts? It's quite easy to use the regular addBolt method and it
will in that case be treated as a stateless one.
Cheers
Alex
On Apr 15, 2016 10:33 AM, "Spico Florin" <sp...@gmail.com> wrote:
> Hello!
> I'm running a topology in LocalCluster that has a stasteful Bolt. Wile
> debugging, I have observed that the initState method is not invoked at all.
> The documentation said:
> "The initState method is invoked by the framework during the bolt
> initialization with the previously saved state of the bolt. This is invoked
> after prepare but before the bolt starts processing any tuples".
>
> Due to this, the state field remains null and I get NPE when I populate it
> with state .put
> Any idea why the initState is not invoked?
> Regards,
> Florin
>
> Here is my code:
>
> public class TimeSeriesStatefulBolt extends
> BaseStatefulBolt<KeyValueState<Long, Map<String, Float>>> {
>
> private KeyValueState<Long, Map<String, Float>> state;
> @Override
> public void initState(KeyValueState<Long, Map<String, Float>> state) {
> this.state = state;
> }
>