You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Guy Khazma <Gu...@ibm.com> on 2019/12/31 13:35:20 UTC

Re: [DISCUSS] Expensive deterministic UDFs

Hi All,

For the use case where the expensive UDF has constant inputs (literals) we
have proposed the following JIRA and PR which calculates the UDF only once
in the driver:
https://issues.apache.org/jira/browse/SPARK-27692
https://github.com/apache/spark/pull/24593

If considering revisiting the optimization for UDFs the above PR can be
considered as well.

Thanks,
Guy



--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org