You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2017/03/23 08:29:41 UTC
[jira] [Commented] (SPARK-20068) Twenty-two column coalesce has
pool performance when codegen is open
[ https://issues.apache.org/jira/browse/SPARK-20068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15937922#comment-15937922 ]
Sean Owen commented on SPARK-20068:
-----------------------------------
I suspect this is because in older Scala versions, after 22 elements you can no longer use tuples. I don't see an argument here that it's so different so as to be a bug?
> Twenty-two column coalesce has pool performance when codegen is open
> --------------------------------------------------------------------
>
> Key: SPARK-20068
> URL: https://issues.apache.org/jira/browse/SPARK-20068
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.5.0
> Reporter: QQShu1
>
> when the coalesce has Twenty-one column the select performance is well, but the Twenty-two column coalesce the select performance is pool.
> I use the 22 column coalesce select commend is that :
> SELECT
> SUM(COALESCE(col1,0)) AS CUSTOM1,
> SUM(COALESCE(col2,0)) AS CUSTOM2,
> SUM(COALESCE(col3,0)) AS CUSTOM3,
> SUM(COALESCE(col4,0)) AS CUSTOM4,
> SUM(COALESCE(col5,0)) AS CUSTOM5,
> SUM(COALESCE(col6,0)) AS CUSTOM6,
> SUM(COALESCE(col7,0)) AS CUSTOM7,
> SUM(COALESCE(col8,0)) AS CUSTOM8,
> SUM(COALESCE(col9,0)) AS CUSTOM9,
> SUM(COALESCE(col10,0)) AS CUSTOM10,
> SUM(COALESCE(col11,0)) AS CUSTOM11,
> SUM(COALESCE(col12,0)) AS CUSTOM12,
> SUM(COALESCE(col13,0)) AS CUSTOM13,
> SUM(COALESCE(col14,0)) AS CUSTOM14,
> SUM(COALESCE(col15,0)) AS CUSTOM15,
> SUM(COALESCE(col16,0)) AS CUSTOM16,
> SUM(COALESCE(col17,0)) AS CUSTOM17,
> SUM(COALESCE(col18,0)) AS CUSTOM18,
> SUM(COALESCE(col19,0)) AS CUSTOM19,
> SUM(COALESCE(col20,0)) AS CUSTOM20,
> SUM(COALESCE(col21,0)) AS CUSTOM21,
> SUM(COALESCE(col22,0)) AS CUSTOM22
> FROM txt_table ;
> the 21 column select commend it that:
> SELECT
> SUM(COALESCE(col1,0)) AS CUSTOM1,
> SUM(COALESCE(col2,0)) AS CUSTOM2,
> SUM(COALESCE(col3,0)) AS CUSTOM3,
> SUM(COALESCE(col4,0)) AS CUSTOM4,
> SUM(COALESCE(col5,0)) AS CUSTOM5,
> SUM(COALESCE(col6,0)) AS CUSTOM6,
> SUM(COALESCE(col7,0)) AS CUSTOM7,
> SUM(COALESCE(col8,0)) AS CUSTOM8,
> SUM(COALESCE(col9,0)) AS CUSTOM9,
> SUM(COALESCE(col10,0)) AS CUSTOM10,
> SUM(COALESCE(col11,0)) AS CUSTOM11,
> SUM(COALESCE(col12,0)) AS CUSTOM12,
> SUM(COALESCE(col13,0)) AS CUSTOM13,
> SUM(COALESCE(col14,0)) AS CUSTOM14,
> SUM(COALESCE(col15,0)) AS CUSTOM15,
> SUM(COALESCE(col16,0)) AS CUSTOM16,
> SUM(COALESCE(col17,0)) AS CUSTOM17,
> SUM(COALESCE(col18,0)) AS CUSTOM18,
> SUM(COALESCE(col19,0)) AS CUSTOM19,
> SUM(COALESCE(col20,0)) AS CUSTOM20,
> SUM(COALESCE(col21,0)) AS CUSTOM21
> FROM txt_table ;
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org