You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Maximilian Sackel (Jira)" <ji...@apache.org> on 2022/04/21 09:50:00 UTC
[jira] [Created] (SPARK-38981) Unexpected commutative property of udf/pandas_udf
Maximilian Sackel created SPARK-38981:
-----------------------------------------
Summary: Unexpected commutative property of udf/pandas_udf
Key: SPARK-38981
URL: https://issues.apache.org/jira/browse/SPARK-38981
Project: Spark
Issue Type: Bug
Components: Optimizer
Affects Versions: 3.2.1
Reporter: Maximilian Sackel
Attachments: screenshot-1.png
Hello all,
When running the attached minmal working example, the order of the filter and the UDF is swapped by the optimizer. This can lead to errors, which are difficult to debug. In the documentation I have found no reference to such behavior.
Is this a bug or a functionality which is poorly documented?
With kind regards,
Max
!image-2022-04-21-11-47-36-931.png!
!image-2022-04-21-11-48-17-966.png!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org