You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by yeikel valdes <em...@yeikel.com> on 2020/02/25 16:23:25 UTC
What options do I have to handle third party classes that are not
serializable?
I am currently using a third party library(Lucene) with Spark that is not serializable. Due to that reason, it generates the following exception :
Job aborted due to stage failure: Task 144.0 in stage 25.0 (TID 2122) had a not serializable result: org.apache.lucene.facet.FacetsConfig Serialization stack: - object not serializable (class: org.apache.lucene.facet.FacetsConfig, value: org.apache.lucene.facet.FacetsConfg
While it would be ideal if this class was serializable, there is really nothing I can do to change this third party library in order to add serialization to it.
What options do I have, and what's the recommended option to handle this problem?
Thank you!
Re: What options do I have to handle third party classes that are not serializable?
Posted by Jeff Evans <je...@gmail.com>.
Did you try this? https://stackoverflow.com/a/2114387/375670
On Tue, Feb 25, 2020 at 10:23 AM yeikel valdes <em...@yeikel.com> wrote:
> I am currently using a third party library(Lucene) with Spark that is not
> serializable. Due to that reason, it generates the following exception :
>
> Job aborted due to stage failure: Task 144.0 in stage 25.0 (TID 2122) had a not serializable result: org.apache.lucene.facet.FacetsConfig Serialization stack: - object not serializable (class: org.apache.lucene.facet.FacetsConfig, value: org.apache.lucene.facet.FacetsConfg
>
> While it would be ideal if this class was serializable, there is really nothing I can do to change this third party library in order to add serialization to it.
>
> What options do I have, and what's the recommended option to handle this problem?
>
> Thank you!
>
>
>
Re: What options do I have to handle third party classes that are not serializable?
Posted by Jeff Evans <je...@gmail.com>.
Did you try this? https://stackoverflow.com/a/2114387/375670
On Tue, Feb 25, 2020 at 10:23 AM yeikel valdes <em...@yeikel.com> wrote:
> I am currently using a third party library(Lucene) with Spark that is not
> serializable. Due to that reason, it generates the following exception :
>
> Job aborted due to stage failure: Task 144.0 in stage 25.0 (TID 2122) had a not serializable result: org.apache.lucene.facet.FacetsConfig Serialization stack: - object not serializable (class: org.apache.lucene.facet.FacetsConfig, value: org.apache.lucene.facet.FacetsConfg
>
> While it would be ideal if this class was serializable, there is really nothing I can do to change this third party library in order to add serialization to it.
>
> What options do I have, and what's the recommended option to handle this problem?
>
> Thank you!
>
>
>