You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2019/05/21 04:38:01 UTC
[jira] [Resolved] (SPARK-16807) Optimize some ABS() statements
[ https://issues.apache.org/jira/browse/SPARK-16807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-16807.
----------------------------------
Resolution: Incomplete
> Optimize some ABS() statements
> ------------------------------
>
> Key: SPARK-16807
> URL: https://issues.apache.org/jira/browse/SPARK-16807
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Sylvain Zimmer
> Priority: Minor
> Labels: bulk-closed
>
> I'm not a Catalyst expert, but I think some use cases for the ABS() function could generate simpler code.
> This is the code generated when doing something like {{ABS(x - y) > 0}} or {{ABS(x - y) = 0}} in Spark SQL:
> {code}
> /* 267 */ float filter_value6 = -1.0f;
> /* 268 */ filter_value6 = agg_value27 - agg_value32;
> /* 269 */ float filter_value5 = -1.0f;
> /* 270 */ filter_value5 = (float)(java.lang.Math.abs(filter_value6));
> /* 271 */
> /* 272 */ boolean filter_value4 = false;
> /* 273 */ filter_value4 = org.apache.spark.util.Utils.nanSafeCompareFloats(filter_value5, 0.0f) > 0;
> /* 274 */ if (!filter_value4) continue;
> {code}
> Maybe it could all be simplified to something like this?
> {code}
> filter_value4 = (agg_value27 != agg_value32)
> {code}
> (Of course you could write {{x != y}} directly in the SQL query, but the {{0}} in my example could be a configurable threshold, not something you can hardcode)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org