[jira] [Created] (SPARK-26201) python broadcast.value on driver fails with disk encryption enabled

["Thomas Graves (JIRA)" <ji...@apache.org>]

Thomas Graves created SPARK-26201:
-------------------------------------

             Summary: python broadcast.value on driver fails with disk encryption enabled
                 Key: SPARK-26201
                 URL: https://issues.apache.org/jira/browse/SPARK-26201
             Project: Spark
          Issue Type: Bug
          Components: PySpark
    Affects Versions: 2.3.2
            Reporter: Thomas Graves


I was trying python with rpc and disk encryption enabled and when I tried a python broadcast variable and just read the value back on the driver side the job failed with:

 

Traceback (most recent call last): File "broadcast.py", line 37, in <module> words_new.value File "/pyspark.zip/pyspark/broadcast.py", line 137, in value File "pyspark.zip/pyspark/broadcast.py", line 122, in load_from_path File "pyspark.zip/pyspark/broadcast.py", line 128, in load EOFError: Ran out of input

To reproduce use configs: --conf spark.network.crypto.enabled=true --conf spark.io.encryption.enabled=true

 

Code:

words_new = sc.broadcast(["scala", "java", "hadoop", "spark", "akka"])
 words_new.value
 print(words_new.value)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org