(spark) branch branch-3.5 updated: [SPARK-56463][PYTHON] Disallow unpickling UDT

[gu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.5 by this push:
     new 092f02df06e8 [SPARK-56463][PYTHON] Disallow unpickling UDT
092f02df06e8 is described below

commit 092f02df06e824168b9078e06d83429f86664299
Author: Tian Gao <ga...@hotmail.com>
AuthorDate: Wed Apr 15 08:20:22 2026 +0900

    [SPARK-56463][PYTHON] Disallow unpickling UDT
    
    ### What changes were proposed in this pull request?
    
    Disallow unpickling UDT.
    
    ### Why are the changes needed?
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes, unpickling UDT is not supported.
    
    ### How was this patch tested?
    
    CI.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No.
    
    Closes #55330 from gaogaotiantian/disallow-unpickling.
    
    Authored-by: Tian Gao <ga...@hotmail.com>
    Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
 python/pyspark/sql/types.py       | 6 ++++--
 python/pyspark/testing/objects.py | 2 +-
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/python/pyspark/sql/types.py b/python/pyspark/sql/types.py
index d4d6b6b086ae..04f430a259c0 100644
--- a/python/pyspark/sql/types.py
+++ b/python/pyspark/sql/types.py
@@ -1204,8 +1204,10 @@ class UserDefinedType(DataType):
         pyClass = pyUDT[split + 1 :]
         m = __import__(pyModule, globals(), locals(), [pyClass])
         if not hasattr(m, pyClass):
-            s = base64.b64decode(json["serializedClass"].encode("utf-8"))
-            UDT = CloudPickleSerializer().loads(s)
+            raise PySparkValueError(
+                errorClass="UNSUPPORTED_OPERATION",
+                messageParameters={"operation": "unpickling user defined types"},
+            )
         else:
             UDT = getattr(m, pyClass)
         return UDT()
diff --git a/python/pyspark/testing/objects.py b/python/pyspark/testing/objects.py
index 5b97664afbdd..2acbd0ba1b41 100644
--- a/python/pyspark/testing/objects.py
+++ b/python/pyspark/testing/objects.py
@@ -90,7 +90,7 @@ class PythonOnlyUDT(UserDefinedType):
 
     @classmethod
     def module(cls):
-        return "__main__"
+        return "pyspark.testing.objects"
 
     def serialize(self, obj):
         return [obj.x, obj.y]


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org