You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by "yangxun@bupt.edu.cn" <ya...@bupt.edu.cn> on 2015/05/20 05:12:17 UTC
StackOverFlow
hi,
I encountered the following error exception:
java.lang.StackOverflowError at java.io.ObjectStreamClass$FieldReflector.getPrimFieldValues(ObjectStreamClass.java:1930) at java.io.ObjectStreamClass.getPrimFieldValues(ObjectStreamClass.java:1233) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1532) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at
…………
I tried to locate the error, but I could hardly figure it out.
I checked all my for loop and while loop, and there is no infinite loop.
all tuples are 'newed' and kept imutable.
how can I figure this out?
thanks ...
ps. this happened both in clusters and local mode.
yangxun@bupt.edu.cn
Re: Re: StackOverFlow
Posted by "yangxun@bupt.edu.cn" <ya...@bupt.edu.cn>.
Sorry I should have used more proper terminology in my original email, the
standard accepted term for this would be a "cycle". An object that has
references to its own type is one scenario where you _could_ have a cycle,
but it does not necessarily mean that you would have one. Here's a simpler
example:
public class Node implements Serializable {
public Node next;
}
// no cycle
Node a = new Node();
Node b = new Node();
a.next = b;
b.next = null;
collector.emit(a); // this is ok
// cycle
Node c = new Node();
Node d = new Node();
c.next = d;
d.next = c;
collector.emit(c); // this is bad
On Wed, May 20, 2015 at 10:54 PM, yangxun@bupt.edu.cn <ya...@bupt.edu.cn>
wrote:
> Yeah, I did some guarantee to keep the data structure imutable just now.
> But I still cannot emit my data structure.
> There must be some problem in my structure, but I don't know what kind of
> lapse would cause such error.
> I got a data structure like this:
> public class Counter implements Serializable{
> private Counter left = null;
> private Counter right = null;
> /*
> the rest
> */
> }
> Is this you what referred to recursive reference?
> Thank you very much.
>
> ------------------------------
> yangxun@bupt.edu.cn
>
> *From:* Nathan Leung <nc...@gmail.com>
> *Date:* 2015-05-21 10:25
> *To:* yangxun@bupt.edu.cn
> *CC:* user <us...@storm.apache.org>
> *Subject:* Re: Re: StackOverFlow
> If the data structure is not immutable then you cannot guarantee that
> it's the same as when your partner emitted it. :) It's hard to say
> without code, and even then, if it's a race condition between bolts I'm
> guessing it won't be easy to diagnose. If you have a fanout (bolt b and
> bolt c both subscribe to bolt a), then you could be in trouble (if tuple
> stays in the same worker for both b and c). If bolt a emits a tuple, then
> modifies the object, you could be in trouble. And whether you get into
> trouble or not, and how much trouble, depends on what you are doing with
> the object.
>
> On Wed, May 20, 2015 at 9:21 PM, yangxun@bupt.edu.cn <ya...@bupt.edu.cn>
> wrote:
>
>> Nathan,
>> You are right ! I'm trying to emitting some data structure.
>> and I'm pretty sure the problem is caused by this.
>> but I can't figure out what is wrong with my data structure...
>> You said that the data structure may be cyclic, but my partner emitted
>> it perfectly,using the same data structure.
>> Do I have to do some serialization myself?
>> Or is some rules I overlooked?
>> If the data structure is not kept imutable, will this error happen too?
>> thank you~
>>
>> ------------------------------
>> yangxun@bupt.edu.cn
>>
>> *From:* Nathan Leung <nc...@gmail.com>
>> *Date:* 2015-05-20 21:12
>> *To:* user <us...@storm.apache.org>
>> *Subject:* Re: StackOverFlow
>> It looks like he's in a recursive loop in a java serialization routine
>> (ObjectOutputStream). This leads me to believe that he is serializing some
>> data structure (perhaps by emitting it in the framework) and since the data
>> structure is cyclic, the stack ends up blown.
>>
>> On Wed, May 20, 2015 at 9:03 AM, Jeffery Maass <ma...@gmail.com> wrote:
>>
>>> Nathan, could you expand on what you mean by "your data structure".
>>>
>>> yangxun@bupt.edu.cn, could you try turning off *Config.TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION
>>> ?* *Config conf = new backtype.storm.Config();*
>>> conf.setFallBackOnJavaSerialization(false); StormSubmitter.submitTopology(topo_name,
>>> conf, builder.createTopology());
>>> If what I think is happening is true, you will receive a different error
>>> in your worker.
>>>
>>>
>>>
>>>
>>> Thank you for your time!
>>>
>>> +++++++++++++++++++++
>>> Jeff Maass <ma...@gmail.com>
>>> linkedin.com/in/jeffmaass
>>> stackoverflow.com/users/373418/maassql
>>> +++++++++++++++++++++
>>>
>>>
>>> On Tue, May 19, 2015 at 10:23 PM, Nathan Leung <nc...@gmail.com>
>>> wrote:
>>>
>>>> Looks like you have a reference loop in your data structure
>>>> On May 19, 2015 11:07 PM, "yangxun@bupt.edu.cn" <ya...@bupt.edu.cn>
>>>> wrote:
>>>>
>>>>> hi,
>>>>>
>>>>> I encountered the following error exception:
>>>>>
>>>>> java.lang.StackOverflowError at
>>>>> java.io.ObjectStreamClass$FieldReflector.getPrimFieldValues(ObjectStreamClass.java:1930)
>>>>> at
>>>>> java.io.ObjectStreamClass.getPrimFieldValues(ObjectStreamClass.java:1233)
>>>>> at
>>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1532)
>>>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>>>> at
>>>>>
>>>>> …………
>>>>>
>>>>> I tried to locate the error, but I could hardly figure it out.
>>>>>
>>>>> I checked all my for loop and while loop, and there is no infinite
>>>>> loop.
>>>>>
>>>>> all tuples are 'newed' and kept imutable.
>>>>>
>>>>> how can I figure this out?
>>>>>
>>>>> thanks ...
>>>>>
>>>>> ps. this happened both in clusters and local mode.
>>>>>
>>>>> ------------------------------
>>>>> yangxun@bupt.edu.cn
>>>>>
>>>>
>>>
>>
>
Re: Re: StackOverFlow
Posted by Nathan Leung <nc...@gmail.com>.
Sorry I should have used more proper terminology in my original email, the
standard accepted term for this would be a "cycle". An object that has
references to its own type is one scenario where you _could_ have a cycle,
but it does not necessarily mean that you would have one. Here's a simpler
example:
public class Node implements Serializable {
public Node next;
}
// no cycle
Node a = new Node();
Node b = new Node();
a.next = b;
b.next = null;
collector.emit(a); // this is ok
// cycle
Node c = new Node();
Node d = new Node();
c.next = d;
d.next = c;
collector.emit(c); // this is bad
On Wed, May 20, 2015 at 10:54 PM, yangxun@bupt.edu.cn <ya...@bupt.edu.cn>
wrote:
> Yeah, I did some guarantee to keep the data structure imutable just now.
> But I still cannot emit my data structure.
> There must be some problem in my structure, but I don't know what kind of
> lapse would cause such error.
> I got a data structure like this:
> public class Counter implements Serializable{
> private Counter left = null;
> private Counter right = null;
> /*
> the rest
> */
> }
> Is this you what referred to recursive reference?
> Thank you very much.
>
> ------------------------------
> yangxun@bupt.edu.cn
>
> *From:* Nathan Leung <nc...@gmail.com>
> *Date:* 2015-05-21 10:25
> *To:* yangxun@bupt.edu.cn
> *CC:* user <us...@storm.apache.org>
> *Subject:* Re: Re: StackOverFlow
> If the data structure is not immutable then you cannot guarantee that
> it's the same as when your partner emitted it. :) It's hard to say
> without code, and even then, if it's a race condition between bolts I'm
> guessing it won't be easy to diagnose. If you have a fanout (bolt b and
> bolt c both subscribe to bolt a), then you could be in trouble (if tuple
> stays in the same worker for both b and c). If bolt a emits a tuple, then
> modifies the object, you could be in trouble. And whether you get into
> trouble or not, and how much trouble, depends on what you are doing with
> the object.
>
> On Wed, May 20, 2015 at 9:21 PM, yangxun@bupt.edu.cn <ya...@bupt.edu.cn>
> wrote:
>
>> Nathan,
>> You are right ! I'm trying to emitting some data structure.
>> and I'm pretty sure the problem is caused by this.
>> but I can't figure out what is wrong with my data structure...
>> You said that the data structure may be cyclic, but my partner emitted
>> it perfectly,using the same data structure.
>> Do I have to do some serialization myself?
>> Or is some rules I overlooked?
>> If the data structure is not kept imutable, will this error happen too?
>> thank you~
>>
>> ------------------------------
>> yangxun@bupt.edu.cn
>>
>> *From:* Nathan Leung <nc...@gmail.com>
>> *Date:* 2015-05-20 21:12
>> *To:* user <us...@storm.apache.org>
>> *Subject:* Re: StackOverFlow
>> It looks like he's in a recursive loop in a java serialization routine
>> (ObjectOutputStream). This leads me to believe that he is serializing some
>> data structure (perhaps by emitting it in the framework) and since the data
>> structure is cyclic, the stack ends up blown.
>>
>> On Wed, May 20, 2015 at 9:03 AM, Jeffery Maass <ma...@gmail.com> wrote:
>>
>>> Nathan, could you expand on what you mean by "your data structure".
>>>
>>> yangxun@bupt.edu.cn, could you try turning off *Config.TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION
>>> ?* *Config conf = new backtype.storm.Config();*
>>> conf.setFallBackOnJavaSerialization(false); StormSubmitter.submitTopology(topo_name,
>>> conf, builder.createTopology());
>>> If what I think is happening is true, you will receive a different error
>>> in your worker.
>>>
>>>
>>>
>>>
>>> Thank you for your time!
>>>
>>> +++++++++++++++++++++
>>> Jeff Maass <ma...@gmail.com>
>>> linkedin.com/in/jeffmaass
>>> stackoverflow.com/users/373418/maassql
>>> +++++++++++++++++++++
>>>
>>>
>>> On Tue, May 19, 2015 at 10:23 PM, Nathan Leung <nc...@gmail.com>
>>> wrote:
>>>
>>>> Looks like you have a reference loop in your data structure
>>>> On May 19, 2015 11:07 PM, "yangxun@bupt.edu.cn" <ya...@bupt.edu.cn>
>>>> wrote:
>>>>
>>>>> hi,
>>>>>
>>>>> I encountered the following error exception:
>>>>>
>>>>> java.lang.StackOverflowError at
>>>>> java.io.ObjectStreamClass$FieldReflector.getPrimFieldValues(ObjectStreamClass.java:1930)
>>>>> at
>>>>> java.io.ObjectStreamClass.getPrimFieldValues(ObjectStreamClass.java:1233)
>>>>> at
>>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1532)
>>>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>>>> at
>>>>>
>>>>> …………
>>>>>
>>>>> I tried to locate the error, but I could hardly figure it out.
>>>>>
>>>>> I checked all my for loop and while loop, and there is no infinite
>>>>> loop.
>>>>>
>>>>> all tuples are 'newed' and kept imutable.
>>>>>
>>>>> how can I figure this out?
>>>>>
>>>>> thanks ...
>>>>>
>>>>> ps. this happened both in clusters and local mode.
>>>>>
>>>>> ------------------------------
>>>>> yangxun@bupt.edu.cn
>>>>>
>>>>
>>>
>>
>
Re: Re: StackOverFlow
Posted by "yangxun@bupt.edu.cn" <ya...@bupt.edu.cn>.
Yeah, I did some guarantee to keep the data structure imutable just now.
But I still cannot emit my data structure.
There must be some problem in my structure, but I don't know what kind of lapse would cause such error.
I got a data structure like this:
public class Counter implements Serializable{
private Counter left = null;
private Counter right = null;
/*
the rest
*/
}
Is this you what referred to recursive reference?
Thank you very much.
yangxun@bupt.edu.cn
From: Nathan Leung
Date: 2015-05-21 10:25
To: yangxun@bupt.edu.cn
CC: user
Subject: Re: Re: StackOverFlow
If the data structure is not immutable then you cannot guarantee that it's the same as when your partner emitted it. :) It's hard to say without code, and even then, if it's a race condition between bolts I'm guessing it won't be easy to diagnose. If you have a fanout (bolt b and bolt c both subscribe to bolt a), then you could be in trouble (if tuple stays in the same worker for both b and c). If bolt a emits a tuple, then modifies the object, you could be in trouble. And whether you get into trouble or not, and how much trouble, depends on what you are doing with the object.
On Wed, May 20, 2015 at 9:21 PM, yangxun@bupt.edu.cn <ya...@bupt.edu.cn> wrote:
Nathan,
You are right ! I'm trying to emitting some data structure.
and I'm pretty sure the problem is caused by this.
but I can't figure out what is wrong with my data structure...
You said that the data structure may be cyclic, but my partner emitted it perfectly,using the same data structure.
Do I have to do some serialization myself?
Or is some rules I overlooked?
If the data structure is not kept imutable, will this error happen too?
thank you~
yangxun@bupt.edu.cn
From: Nathan Leung
Date: 2015-05-20 21:12
To: user
Subject: Re: StackOverFlow
It looks like he's in a recursive loop in a java serialization routine (ObjectOutputStream). This leads me to believe that he is serializing some data structure (perhaps by emitting it in the framework) and since the data structure is cyclic, the stack ends up blown.
On Wed, May 20, 2015 at 9:03 AM, Jeffery Maass <ma...@gmail.com> wrote:
Nathan, could you expand on what you mean by "your data structure".
yangxun@bupt.edu.cn, could you try turning off Config.TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION ?
Config conf = new backtype.storm.Config();
conf.setFallBackOnJavaSerialization(false);
StormSubmitter.submitTopology(topo_name, conf, builder.createTopology());
If what I think is happening is true, you will receive a different error in your worker.
Thank you for your time!
+++++++++++++++++++++
Jeff Maass
linkedin.com/in/jeffmaass
stackoverflow.com/users/373418/maassql
+++++++++++++++++++++
On Tue, May 19, 2015 at 10:23 PM, Nathan Leung <nc...@gmail.com> wrote:
Looks like you have a reference loop in your data structure
On May 19, 2015 11:07 PM, "yangxun@bupt.edu.cn" <ya...@bupt.edu.cn> wrote:
hi,
I encountered the following error exception:
java.lang.StackOverflowError at java.io.ObjectStreamClass$FieldReflector.getPrimFieldValues(ObjectStreamClass.java:1930) at java.io.ObjectStreamClass.getPrimFieldValues(ObjectStreamClass.java:1233) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1532) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at
…………
I tried to locate the error, but I could hardly figure it out.
I checked all my for loop and while loop, and there is no infinite loop.
all tuples are 'newed' and kept imutable.
how can I figure this out?
thanks ...
ps. this happened both in clusters and local mode.
yangxun@bupt.edu.cn
Re: Re: StackOverFlow
Posted by Nathan Leung <nc...@gmail.com>.
If the data structure is not immutable then you cannot guarantee that it's
the same as when your partner emitted it. :) It's hard to say without
code, and even then, if it's a race condition between bolts I'm guessing it
won't be easy to diagnose. If you have a fanout (bolt b and bolt c both
subscribe to bolt a), then you could be in trouble (if tuple stays in the
same worker for both b and c). If bolt a emits a tuple, then modifies the
object, you could be in trouble. And whether you get into trouble or not,
and how much trouble, depends on what you are doing with the object.
On Wed, May 20, 2015 at 9:21 PM, yangxun@bupt.edu.cn <ya...@bupt.edu.cn>
wrote:
> Nathan,
> You are right ! I'm trying to emitting some data structure.
> and I'm pretty sure the problem is caused by this.
> but I can't figure out what is wrong with my data structure...
> You said that the data structure may be cyclic, but my partner emitted
> it perfectly,using the same data structure.
> Do I have to do some serialization myself?
> Or is some rules I overlooked?
> If the data structure is not kept imutable, will this error happen too?
> thank you~
>
> ------------------------------
> yangxun@bupt.edu.cn
>
> *From:* Nathan Leung <nc...@gmail.com>
> *Date:* 2015-05-20 21:12
> *To:* user <us...@storm.apache.org>
> *Subject:* Re: StackOverFlow
> It looks like he's in a recursive loop in a java serialization routine
> (ObjectOutputStream). This leads me to believe that he is serializing some
> data structure (perhaps by emitting it in the framework) and since the data
> structure is cyclic, the stack ends up blown.
>
> On Wed, May 20, 2015 at 9:03 AM, Jeffery Maass <ma...@gmail.com> wrote:
>
>> Nathan, could you expand on what you mean by "your data structure".
>>
>> yangxun@bupt.edu.cn, could you try turning off *Config.TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION
>> ?* *Config conf = new backtype.storm.Config();*
>> conf.setFallBackOnJavaSerialization(false); StormSubmitter.submitTopology(topo_name,
>> conf, builder.createTopology());
>> If what I think is happening is true, you will receive a different error
>> in your worker.
>>
>>
>>
>>
>> Thank you for your time!
>>
>> +++++++++++++++++++++
>> Jeff Maass <ma...@gmail.com>
>> linkedin.com/in/jeffmaass
>> stackoverflow.com/users/373418/maassql
>> +++++++++++++++++++++
>>
>>
>> On Tue, May 19, 2015 at 10:23 PM, Nathan Leung <nc...@gmail.com> wrote:
>>
>>> Looks like you have a reference loop in your data structure
>>> On May 19, 2015 11:07 PM, "yangxun@bupt.edu.cn" <ya...@bupt.edu.cn>
>>> wrote:
>>>
>>>> hi,
>>>>
>>>> I encountered the following error exception:
>>>>
>>>> java.lang.StackOverflowError at
>>>> java.io.ObjectStreamClass$FieldReflector.getPrimFieldValues(ObjectStreamClass.java:1930)
>>>> at
>>>> java.io.ObjectStreamClass.getPrimFieldValues(ObjectStreamClass.java:1233)
>>>> at
>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1532)
>>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>>> at
>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>>> at
>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>>> at
>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>>> at
>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>>> at
>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>>> at
>>>>
>>>> …………
>>>>
>>>> I tried to locate the error, but I could hardly figure it out.
>>>>
>>>> I checked all my for loop and while loop, and there is no infinite loop.
>>>>
>>>> all tuples are 'newed' and kept imutable.
>>>>
>>>> how can I figure this out?
>>>>
>>>> thanks ...
>>>>
>>>> ps. this happened both in clusters and local mode.
>>>>
>>>> ------------------------------
>>>> yangxun@bupt.edu.cn
>>>>
>>>
>>
>
Re: Re: StackOverFlow
Posted by "yangxun@bupt.edu.cn" <ya...@bupt.edu.cn>.
It looks like he's in a recursive loop in a java serialization routine
(ObjectOutputStream). This leads me to believe that he is serializing some
data structure (perhaps by emitting it in the framework) and since the data
structure is cyclic, the stack ends up blown.
On Wed, May 20, 2015 at 9:03 AM, Jeffery Maass <ma...@gmail.com> wrote:
> Nathan, could you expand on what you mean by "your data structure".
>
> yangxun@bupt.edu.cn, could you try turning off *Config.TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION
> ?**Config conf = new backtype.storm.Config();*
> conf.setFallBackOnJavaSerialization(false);StormSubmitter.submitTopology(topo_name,
> conf, builder.createTopology());
> If what I think is happening is true, you will receive a different error
> in your worker.
>
>
>
>
> Thank you for your time!
>
> +++++++++++++++++++++
> Jeff Maass <ma...@gmail.com>
> linkedin.com/in/jeffmaass
> stackoverflow.com/users/373418/maassql
> +++++++++++++++++++++
>
>
> On Tue, May 19, 2015 at 10:23 PM, Nathan Leung <nc...@gmail.com> wrote:
>
>> Looks like you have a reference loop in your data structure
>> On May 19, 2015 11:07 PM, "yangxun@bupt.edu.cn" <ya...@bupt.edu.cn>
>> wrote:
>>
>>> hi,
>>>
>>> I encountered the following error exception:
>>>
>>> java.lang.StackOverflowError at
>>> java.io.ObjectStreamClass$FieldReflector.getPrimFieldValues(ObjectStreamClass.java:1930)
>>> at
>>> java.io.ObjectStreamClass.getPrimFieldValues(ObjectStreamClass.java:1233)
>>> at
>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1532)
>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>> at
>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>> at
>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>> at
>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>> at
>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>> at
>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>> at
>>>
>>> …………
>>>
>>> I tried to locate the error, but I could hardly figure it out.
>>>
>>> I checked all my for loop and while loop, and there is no infinite loop.
>>>
>>> all tuples are 'newed' and kept imutable.
>>>
>>> how can I figure this out?
>>>
>>> thanks ...
>>>
>>> ps. this happened both in clusters and local mode.
>>>
>>> ------------------------------
>>> yangxun@bupt.edu.cn
>>>
>>
>
Re: StackOverFlow
Posted by Nathan Leung <nc...@gmail.com>.
It looks like he's in a recursive loop in a java serialization routine
(ObjectOutputStream). This leads me to believe that he is serializing some
data structure (perhaps by emitting it in the framework) and since the data
structure is cyclic, the stack ends up blown.
On Wed, May 20, 2015 at 9:03 AM, Jeffery Maass <ma...@gmail.com> wrote:
> Nathan, could you expand on what you mean by "your data structure".
>
> yangxun@bupt.edu.cn, could you try turning off *Config.TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION
> ?**Config conf = new backtype.storm.Config();*
> conf.setFallBackOnJavaSerialization(false);StormSubmitter.submitTopology(topo_name,
> conf, builder.createTopology());
> If what I think is happening is true, you will receive a different error
> in your worker.
>
>
>
>
> Thank you for your time!
>
> +++++++++++++++++++++
> Jeff Maass <ma...@gmail.com>
> linkedin.com/in/jeffmaass
> stackoverflow.com/users/373418/maassql
> +++++++++++++++++++++
>
>
> On Tue, May 19, 2015 at 10:23 PM, Nathan Leung <nc...@gmail.com> wrote:
>
>> Looks like you have a reference loop in your data structure
>> On May 19, 2015 11:07 PM, "yangxun@bupt.edu.cn" <ya...@bupt.edu.cn>
>> wrote:
>>
>>> hi,
>>>
>>> I encountered the following error exception:
>>>
>>> java.lang.StackOverflowError at
>>> java.io.ObjectStreamClass$FieldReflector.getPrimFieldValues(ObjectStreamClass.java:1930)
>>> at
>>> java.io.ObjectStreamClass.getPrimFieldValues(ObjectStreamClass.java:1233)
>>> at
>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1532)
>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>> at
>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>> at
>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>> at
>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>> at
>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>>> at
>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>>> at
>>>
>>> …………
>>>
>>> I tried to locate the error, but I could hardly figure it out.
>>>
>>> I checked all my for loop and while loop, and there is no infinite loop.
>>>
>>> all tuples are 'newed' and kept imutable.
>>>
>>> how can I figure this out?
>>>
>>> thanks ...
>>>
>>> ps. this happened both in clusters and local mode.
>>>
>>> ------------------------------
>>> yangxun@bupt.edu.cn
>>>
>>
>
Re: StackOverFlow
Posted by Jeffery Maass <ma...@gmail.com>.
Nathan, could you expand on what you mean by "your data structure".
yangxun@bupt.edu.cn, could you try turning off
*Config.TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION
?**Config conf = new backtype.storm.Config();*
conf.setFallBackOnJavaSerialization(false);StormSubmitter.submitTopology(topo_name,
conf, builder.createTopology());
If what I think is happening is true, you will receive a different error in
your worker.
Thank you for your time!
+++++++++++++++++++++
Jeff Maass <ma...@gmail.com>
linkedin.com/in/jeffmaass
stackoverflow.com/users/373418/maassql
+++++++++++++++++++++
On Tue, May 19, 2015 at 10:23 PM, Nathan Leung <nc...@gmail.com> wrote:
> Looks like you have a reference loop in your data structure
> On May 19, 2015 11:07 PM, "yangxun@bupt.edu.cn" <ya...@bupt.edu.cn>
> wrote:
>
>> hi,
>>
>> I encountered the following error exception:
>>
>> java.lang.StackOverflowError at
>> java.io.ObjectStreamClass$FieldReflector.getPrimFieldValues(ObjectStreamClass.java:1930)
>> at
>> java.io.ObjectStreamClass.getPrimFieldValues(ObjectStreamClass.java:1233)
>> at
>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1532)
>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>> at
>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>> at
>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>> at
>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>> at
>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>> at
>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>> at
>>
>> …………
>>
>> I tried to locate the error, but I could hardly figure it out.
>>
>> I checked all my for loop and while loop, and there is no infinite loop.
>>
>> all tuples are 'newed' and kept imutable.
>>
>> how can I figure this out?
>>
>> thanks ...
>>
>> ps. this happened both in clusters and local mode.
>>
>> ------------------------------
>> yangxun@bupt.edu.cn
>>
>
Re: StackOverFlow
Posted by Nathan Leung <nc...@gmail.com>.
Looks like you have a reference loop in your data structure
On May 19, 2015 11:07 PM, "yangxun@bupt.edu.cn" <ya...@bupt.edu.cn> wrote:
> hi,
>
> I encountered the following error exception:
>
> java.lang.StackOverflowError at
> java.io.ObjectStreamClass$FieldReflector.getPrimFieldValues(ObjectStreamClass.java:1930)
> at
> java.io.ObjectStreamClass.getPrimFieldValues(ObjectStreamClass.java:1233)
> at
> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1532)
> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> at
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> at
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> at
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> at
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> at
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> at
>
> …………
>
> I tried to locate the error, but I could hardly figure it out.
>
> I checked all my for loop and while loop, and there is no infinite loop.
>
> all tuples are 'newed' and kept imutable.
>
> how can I figure this out?
>
> thanks ...
>
> ps. this happened both in clusters and local mode.
>
> ------------------------------
> yangxun@bupt.edu.cn
>