(spark) branch master updated: [SPARK-47858][PYTHON][FOLLOWUP] Excluding Python magic methods from error context target

[gu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 2e5825fb32c0 [SPARK-47858][PYTHON][FOLLOWUP] Excluding Python magic methods from error context target
2e5825fb32c0 is described below

commit 2e5825fb32c03faa854921c54feb6ad0e7b5d432
Author: Haejoon Lee <ha...@databricks.com>
AuthorDate: Fri Apr 26 16:53:22 2024 +0900

    [SPARK-47858][PYTHON][FOLLOWUP] Excluding Python magic methods from error context target
    
    ### What changes were proposed in this pull request?
    
    This PR followups for https://github.com/apache/spark/pull/46063 to exclude Python magic methods from error context target
    
    ### Why are the changes needed?
    
    We only need to logging the Apache Spark Column API functions for DataFrameQueryContext, but not Python native functions.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No API changes, but the error message only contain Column APIs that are supported by Apache Spark
    
    ### How was this patch tested?
    
    The existing CI should pass.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No.
    
    Closes #46215 from itholic/47858-followup.
    
    Authored-by: Haejoon Lee <ha...@databricks.com>
    Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
 python/pyspark/errors/utils.py | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/python/pyspark/errors/utils.py b/python/pyspark/errors/utils.py
index 08744dc4c264..180bdd681306 100644
--- a/python/pyspark/errors/utils.py
+++ b/python/pyspark/errors/utils.py
@@ -197,6 +197,14 @@ def with_origin_to_class(cls: Type[T]) -> Type[T]:
     """
     if os.environ.get("PYSPARK_PIN_THREAD", "true").lower() == "true":
         for name, method in cls.__dict__.items():
-            if callable(method) and name != "__init__":
+            # Excluding Python magic methods that do not utilize JVM functions.
+            if callable(method) and name not in (
+                "__init__",
+                "__new__",
+                "__iter__",
+                "__nonzero__",
+                "__repr__",
+                "__bool__",
+            ):
                 setattr(cls, name, _with_origin(method))
     return cls


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org