[jira] [Commented] (SPARK-15985) Reduce runtime overhead of a program that reads an primitive array in Dataset

["Apache Spark (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-15985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15333331#comment-15333331 ] 

Apache Spark commented on SPARK-15985:
--------------------------------------

User 'kiszk' has created a pull request for this issue:
https://github.com/apache/spark/pull/13704

> Reduce runtime overhead of a program that reads an primitive array in Dataset
> -----------------------------------------------------------------------------
>
>                 Key: SPARK-15985
>                 URL: https://issues.apache.org/jira/browse/SPARK-15985
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>            Reporter: Kazuaki Ishizaki
>
> When a program read an array in Dataset, the code generator create some copy operations. If an array is for primitive type, there are some opportunities for optimizations in generated code to reduce runtime overhead.
> {code}
> val ds = Seq(Array(1.0, 2.0, 3.0), Array(4.0, 5.0, 6.0)).toDS()
> ds.map(p => {
>      var s = 0.0
>      for (i <- 0 to 2) { s += p(i) }
>      s
>    }).show
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org