(spark) branch branch-4.0 updated: [SPARK-53424][PYTHON][TESTS] Hide traceback in `assertSchemaEqual/assertDataFrameEqual/assert_rows_equal`

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.0 by this push:
     new a7884a396956 [SPARK-53424][PYTHON][TESTS] Hide traceback in `assertSchemaEqual/assertDataFrameEqual/assert_rows_equal`
a7884a396956 is described below

commit a7884a3969563ff780194d06bfa752d193c57103
Author: Allison Wang <al...@databricks.com>
AuthorDate: Sat Aug 30 13:14:22 2025 -0700

    [SPARK-53424][PYTHON][TESTS] Hide traceback in `assertSchemaEqual/assertDataFrameEqual/assert_rows_equal`
    
    ### What changes were proposed in this pull request?
    
    This PR hides traceback for several utility functions in testing utils.
    
    ### Why are the changes needed?
    
    To hide unnecessary information when using testing utils like assertDataFrameEqual. The error message is too verbose. For example we do not need to show this `assert_rows_equal` function body:
    ```
     python/pyspark/testing/utils.py:1114: in assertDataFrameEqual
          assert_rows_equal(actual_list, expected_list, maxErrors=maxErrors, showOnlyDiff=showOnlyDiff)
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
          def assert_rows_equal(
              rows1: List[Row], rows2: List[Row], maxErrors: int = None, showOnlyDiff: bool = False
          ):
              zipped = list(zip_longest(rows1, rows2))
              diff_rows_cnt = 0
              diff_rows = []
              has_diff_rows = False
    
              rows_str1 = ""
              rows_str2 = ""
    
              # count different rows
              for r1, r2 in zipped:
                  if not compare_rows(r1, r2):
    ...
    ```
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Manual testing
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No
    
    Closes #52129 from allisonwang-db/minor-traceback.
    
    Authored-by: Allison Wang <al...@databricks.com>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
    (cherry picked from commit 7e7380ff590bcdf498fb49cd623857415be08c10)
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 python/pyspark/testing/utils.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/python/pyspark/testing/utils.py b/python/pyspark/testing/utils.py
index 780f0f90ac62..4d37fd11d359 100644
--- a/python/pyspark/testing/utils.py
+++ b/python/pyspark/testing/utils.py
@@ -387,6 +387,7 @@ def assertSchemaEqual(
     ignoreColumnOrder: bool = False,
     ignoreColumnName: bool = False,
 ):
+    __tracebackhide__ = True
     r"""
     A util function to assert equality between DataFrame schemas `actual` and `expected`.
 
@@ -573,6 +574,7 @@ def assertDataFrameEqual(
     showOnlyDiff: bool = False,
     includeDiffRows=False,
 ):
+    __tracebackhide__ = True
     r"""
     A util function to assert equality between `actual` and `expected`
     (DataFrames or lists of Rows), with optional parameters `checkRowOrder`, `rtol`, and `atol`.
@@ -959,6 +961,7 @@ def assertDataFrameEqual(
     def assert_rows_equal(
         rows1: List[Row], rows2: List[Row], maxErrors: int = None, showOnlyDiff: bool = False
     ):
+        __tracebackhide__ = True
         zipped = list(zip_longest(rows1, rows2))
         diff_rows_cnt = 0
         diff_rows = []


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org