[jira] [Commented] (SPARK-30424) Change ExpressionEncoder toRow method to return UnsafeRow

["Erik Erlandson (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-30424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17015183#comment-17015183 ] 

Erik Erlandson commented on SPARK-30424:
----------------------------------------

The main place this change causes a compile fail on is in SparkSession:

 
{code:java}
def createDataFrame(rowRDD: RDD[Row], schema: StructType): DataFrame{code}
And the key RDD impacted is LogicalRDD.

What I'm wondering is whether it is appropriate to change the signature of the RDD in LogicalRDD from InternalRow to the more specific UnsafeRow. My intuition is no, however it's also true that this is what's actually occurring under the hood currently, so I'm curious what the catalyst maintainers think about it.

 

 

> Change ExpressionEncoder toRow method to return UnsafeRow
> ---------------------------------------------------------
>
>                 Key: SPARK-30424
>                 URL: https://issues.apache.org/jira/browse/SPARK-30424
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.0.0
>            Reporter: Erik Erlandson
>            Priority: Minor
>
> [~wenchen] observed that the toRow() method on ExpressionEncoder can have its return type specified as UnsafeRow. See discussion on [https://github.com/apache/spark/pull/25024] 
>  
> Not a high priority but could be done for 3.0.0



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org