[jira] [Created] (SPARK-58575) Inline redundant error_class parameter in worker.py verify helpers

["Yicong Huang (Jira)" <ji...@apache.org>]

Yicong Huang created SPARK-58575:
------------------------------------

             Summary: Inline redundant error_class parameter in worker.py verify helpers
                 Key: SPARK-58575
                 URL: https://issues.apache.org/jira/browse/SPARK-58575
             Project: Spark
          Issue Type: Improvement
          Components: PySpark
    Affects Versions: 5.0.0
            Reporter: Yicong Huang


Follow-up to SPARK-58529, which consolidated the row-count verification helpers in python/pyspark/worker.py but left two of them over-parameterized with an errorClass argument that is always passed the same constant.

- verify_output_row_limit: both call sites (SQL_SCALAR_ARROW_ITER_UDF and SQL_SCALAR_PANDAS_ITER_UDF) pass error_class="OUTPUT_EXCEEDS_INPUT_ROWS".
- verify_iterator_exhausted: both call sites pass error_class="INPUT_NOT_FULLY_CONSUMED".

The parameter is never used with a different value. These two error classes are semantically bound to these two helpers (the two halves of the iterator-UDF "output rows must equal input rows" contract); no other code path can reuse them. UDTFs, for example, have no such contract and use a different set of verifiers.

Drop the error_class parameter from each signature and inline the constant into the raise. Pure internal simplification: the raised error class and message are unchanged, so there is no user-facing behavior change, no error-conditions.json change, and no test assertion change.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org