You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Juan Rodríguez Hortalá <ju...@gmail.com> on 2014/07/28 17:51:07 UTC

Debugging "Task not serializable"

Hi all,

I was wondering if someone has conceived a method for debugging "Task not
serializable: java.io.NotSerializableException" errors, apart from
commenting and uncommenting parts of the program, or just turning
everything into Serializable. I find this kind of error very hard to debug,
as these are originated in the Spark runtime system.

I'm using Spark for Java.

Thanks a lot in advance,

Juan

Re: Debugging "Task not serializable"

Posted by Juan Rodríguez Hortalá <ju...@gmail.com>.
Hi Sourav,

I will take a look to that too, thanks a lot for your help

Greetings,

Juan


2014-07-30 10:58 GMT+02:00 Sourav Chandra <so...@livestream.com>:

> While running application set this -Dsun.io.serialization.extendedDebugInfo=true
> This is applciable post java 1.6 version
>
>
> On Wed, Jul 30, 2014 at 2:13 PM, Juan Rodríguez Hortalá <
> juan.rodriguez.hortala@gmail.com> wrote:
>
>> Akhil, Andry, thanks a lot for your suggestions. I will take a look to
>> those JVM options.
>>
>> Greetings,
>>
>> Juan
>>
>>
>> 2014-07-28 18:56 GMT+02:00 andy petrella <an...@gmail.com>:
>>
>> Also check the guides for the JVM option that prints messages for such
>>> problems.
>>> Sorry, sent from phone and don't know it by heart :/
>>> Le 28 juil. 2014 18:44, "Akhil Das" <ak...@sigmoidanalytics.com> a
>>> écrit :
>>>
>>>  A quick fix would be to implement java.io.Serializable in those
>>>> classes which are causing this exception.
>>>>
>>>>
>>>>
>>>> Thanks
>>>> Best Regards
>>>>
>>>>
>>>> On Mon, Jul 28, 2014 at 9:21 PM, Juan Rodríguez Hortalá <
>>>> juan.rodriguez.hortala@gmail.com> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I was wondering if someone has conceived a method for debugging "Task
>>>>> not serializable: java.io.NotSerializableException" errors, apart from
>>>>> commenting and uncommenting parts of the program, or just turning
>>>>> everything into Serializable. I find this kind of error very hard to debug,
>>>>> as these are originated in the Spark runtime system.
>>>>>
>>>>> I'm using Spark for Java.
>>>>>
>>>>> Thanks a lot in advance,
>>>>>
>>>>> Juan
>>>>>
>>>>
>>>>
>>
>
>
> --
>
> Sourav Chandra
>
> Senior Software Engineer
>
> · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·
>
> sourav.chandra@livestream.com
>
> o: +91 80 4121 8723
>
> m: +91 988 699 3746
>
> skype: sourav.chandra
>
> Livestream
>
> "Ajmera Summit", First Floor, #3/D, 68 Ward, 3rd Cross, 7th C Main, 3rd
> Block, Koramangala Industrial Area,
>
> Bangalore 560034
>
> www.livestream.com
>

Re: Debugging "Task not serializable"

Posted by Sourav Chandra <so...@livestream.com>.
While running application set this
-Dsun.io.serialization.extendedDebugInfo=true
This is applciable post java 1.6 version


On Wed, Jul 30, 2014 at 2:13 PM, Juan Rodríguez Hortalá <
juan.rodriguez.hortala@gmail.com> wrote:

> Akhil, Andry, thanks a lot for your suggestions. I will take a look to
> those JVM options.
>
> Greetings,
>
> Juan
>
>
> 2014-07-28 18:56 GMT+02:00 andy petrella <an...@gmail.com>:
>
> Also check the guides for the JVM option that prints messages for such
>> problems.
>> Sorry, sent from phone and don't know it by heart :/
>> Le 28 juil. 2014 18:44, "Akhil Das" <ak...@sigmoidanalytics.com> a
>> écrit :
>>
>>  A quick fix would be to implement java.io.Serializable in those classes
>>> which are causing this exception.
>>>
>>>
>>>
>>> Thanks
>>> Best Regards
>>>
>>>
>>> On Mon, Jul 28, 2014 at 9:21 PM, Juan Rodríguez Hortalá <
>>> juan.rodriguez.hortala@gmail.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I was wondering if someone has conceived a method for debugging "Task
>>>> not serializable: java.io.NotSerializableException" errors, apart from
>>>> commenting and uncommenting parts of the program, or just turning
>>>> everything into Serializable. I find this kind of error very hard to debug,
>>>> as these are originated in the Spark runtime system.
>>>>
>>>> I'm using Spark for Java.
>>>>
>>>> Thanks a lot in advance,
>>>>
>>>> Juan
>>>>
>>>
>>>
>


-- 

Sourav Chandra

Senior Software Engineer

· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·

sourav.chandra@livestream.com

o: +91 80 4121 8723

m: +91 988 699 3746

skype: sourav.chandra

Livestream

"Ajmera Summit", First Floor, #3/D, 68 Ward, 3rd Cross, 7th C Main, 3rd
Block, Koramangala Industrial Area,

Bangalore 560034

www.livestream.com

Re: Debugging "Task not serializable"

Posted by Juan Rodríguez Hortalá <ju...@gmail.com>.
Akhil, Andry, thanks a lot for your suggestions. I will take a look to
those JVM options.

Greetings,

Juan


2014-07-28 18:56 GMT+02:00 andy petrella <an...@gmail.com>:

> Also check the guides for the JVM option that prints messages for such
> problems.
> Sorry, sent from phone and don't know it by heart :/
> Le 28 juil. 2014 18:44, "Akhil Das" <ak...@sigmoidanalytics.com> a écrit :
>
> A quick fix would be to implement java.io.Serializable in those classes
>> which are causing this exception.
>>
>>
>>
>> Thanks
>> Best Regards
>>
>>
>> On Mon, Jul 28, 2014 at 9:21 PM, Juan Rodríguez Hortalá <
>> juan.rodriguez.hortala@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I was wondering if someone has conceived a method for debugging "Task
>>> not serializable: java.io.NotSerializableException" errors, apart from
>>> commenting and uncommenting parts of the program, or just turning
>>> everything into Serializable. I find this kind of error very hard to debug,
>>> as these are originated in the Spark runtime system.
>>>
>>> I'm using Spark for Java.
>>>
>>> Thanks a lot in advance,
>>>
>>> Juan
>>>
>>
>>

Re: Debugging "Task not serializable"

Posted by andy petrella <an...@gmail.com>.
Also check the guides for the JVM option that prints messages for such
problems.
Sorry, sent from phone and don't know it by heart :/
Le 28 juil. 2014 18:44, "Akhil Das" <ak...@sigmoidanalytics.com> a écrit :

> A quick fix would be to implement java.io.Serializable in those classes
> which are causing this exception.
>
>
>
> Thanks
> Best Regards
>
>
> On Mon, Jul 28, 2014 at 9:21 PM, Juan Rodríguez Hortalá <
> juan.rodriguez.hortala@gmail.com> wrote:
>
>> Hi all,
>>
>> I was wondering if someone has conceived a method for debugging "Task not
>> serializable: java.io.NotSerializableException" errors, apart from
>> commenting and uncommenting parts of the program, or just turning
>> everything into Serializable. I find this kind of error very hard to debug,
>> as these are originated in the Spark runtime system.
>>
>> I'm using Spark for Java.
>>
>> Thanks a lot in advance,
>>
>> Juan
>>
>
>

Re: Debugging "Task not serializable"

Posted by Akhil Das <ak...@sigmoidanalytics.com>.
A quick fix would be to implement java.io.Serializable in those classes
which are causing this exception.



Thanks
Best Regards


On Mon, Jul 28, 2014 at 9:21 PM, Juan Rodríguez Hortalá <
juan.rodriguez.hortala@gmail.com> wrote:

> Hi all,
>
> I was wondering if someone has conceived a method for debugging "Task not
> serializable: java.io.NotSerializableException" errors, apart from
> commenting and uncommenting parts of the program, or just turning
> everything into Serializable. I find this kind of error very hard to debug,
> as these are originated in the Spark runtime system.
>
> I'm using Spark for Java.
>
> Thanks a lot in advance,
>
> Juan
>