[jira] [Created] (SPARK-30270) Can't pickle abstract classes (with cloudpickle)

["Sebastian Straub (Jira)" <ji...@apache.org>]

Sebastian Straub created SPARK-30270:
----------------------------------------

             Summary: Can't pickle abstract classes (with cloudpickle)
                 Key: SPARK-30270
                 URL: https://issues.apache.org/jira/browse/SPARK-30270
             Project: Spark
          Issue Type: Bug
          Components: PySpark
    Affects Versions: 2.4.4
            Reporter: Sebastian Straub


I can't use any classes that are derived from abstract classes in PySpark, because cloudpickle  can't pickle them. Example:

 
{code:java}
import pickle
from abc import ABC
from pyspark import cloudpickle


class Foo(ABC):
    pass

class Bar(Foo):
    pass

bar = Bar()

# pickle dump works fine
pickle.dumps(bar)
# cloudpickle doesn't
cloudpickle.dumps(bar)
{code}
A similar bug has already been reported in SPARK-21439 and marked resolved, but I can confirm that the issue still persists.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org