You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2022/03/16 01:40:00 UTC
[jira] [Commented] (SPARK-38558) Remove unnecessary casts between IntegerType and IntDecimal
[ https://issues.apache.org/jira/browse/SPARK-38558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17507300#comment-17507300 ]
Apache Spark commented on SPARK-38558:
--------------------------------------
User 'cashmand' has created a pull request for this issue:
https://github.com/apache/spark/pull/35863
> Remove unnecessary casts between IntegerType and IntDecimal
> -----------------------------------------------------------
>
> Key: SPARK-38558
> URL: https://issues.apache.org/jira/browse/SPARK-38558
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.2.1
> Reporter: David Cashman
> Priority: Major
>
> In {{{}NTile{}}}, the number of rows per bucket is computed as {{{}n / buckets{}}}, where {{n}} is the partition size, and {{buckets}} is the argument to {{NTile}} (number of buckets). The code currently casts the arguments to IntDecimal, then casts the result back to IntegerType. This is unnecessary, since it is equivalent to just doing integer division, i.e. {{{}n div buckets{}}}.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org