[jira] [Commented] (SPARK-51392) Improve PythonException for unexpected UDF usage

["rishavsinha (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-51392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18082930#comment-18082930 ] 

rishavsinha commented on SPARK-51392:
-------------------------------------

I investigated this issue on the current master branch.

It appears the issue is already fixed. There is already a regression test covering this case in {{{}HiveDDLSuite{}}}:

{{test("create table - partition column duplicates EMPTY_DATA_SCHEMA")}}

and {{HiveExternalCatalog}} also contains validation logic for conflicting partition column names with {{{}EMPTY_DATA_SCHEMA{}}}.

I was not able to reproduce the issue on current master.

> Improve PythonException for unexpected UDF usage
> ------------------------------------------------
>
>                 Key: SPARK-51392
>                 URL: https://issues.apache.org/jira/browse/SPARK-51392
>             Project: Spark
>          Issue Type: Bug
>          Components: PySpark
>    Affects Versions: 4.0.0
>            Reporter: Haejoon Lee
>            Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org