[jira] [Resolved] (SPARK-15985) Reduce runtime overhead of a program that reads an primitive array in Dataset

["Wenchen Fan (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-15985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wenchen Fan resolved SPARK-15985.
---------------------------------
       Resolution: Fixed
    Fix Version/s: 2.1.0

Issue resolved by pull request 13704
[https://github.com/apache/spark/pull/13704]

> Reduce runtime overhead of a program that reads an primitive array in Dataset
> -----------------------------------------------------------------------------
>
>                 Key: SPARK-15985
>                 URL: https://issues.apache.org/jira/browse/SPARK-15985
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>            Reporter: Kazuaki Ishizaki
>            Assignee: Kazuaki Ishizaki
>             Fix For: 2.1.0
>
>
> When a program read an array in Dataset, the code generator create some copy operations. If an array is for primitive type, there are some opportunities for optimizations in generated code to reduce runtime overhead.
> {code}
> val ds = Seq(Array(1.0, 2.0, 3.0), Array(4.0, 5.0, 6.0)).toDS()
> ds.map(p => {
>      var s = 0.0
>      for (i <- 0 to 2) { s += p(i) }
>      s
>    }).show
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org