You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2015/05/26 04:34:17 UTC

[jira] [Commented] (SPARK-7858) DataSourceStrategy.createPhysicalRDD should use output schema when performing row conversions, not relation schema

    [ https://issues.apache.org/jira/browse/SPARK-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14558626#comment-14558626 ] 

Apache Spark commented on SPARK-7858:
-------------------------------------

User 'JoshRosen' has created a pull request for this issue:
https://github.com/apache/spark/pull/6400

> DataSourceStrategy.createPhysicalRDD should use output schema when performing row conversions, not relation schema
> ------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-7858
>                 URL: https://issues.apache.org/jira/browse/SPARK-7858
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.4.0
>            Reporter: Josh Rosen
>            Assignee: Josh Rosen
>
> In {{DataSourceStrategy.createPhysicalRDD}}, we use the relation schema as the target schema for converting incoming rows into Catalyst rows.  However, we should be using the output schema instead, since our scan might be returning fewer columns due to partition pruning.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org