You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "William Wong (JIRA)" <ji...@apache.org> on 2019/03/28 07:32:00 UTC
[jira] [Comment Edited] (SPARK-24130) Data Source V2: Join Push
Down
[ https://issues.apache.org/jira/browse/SPARK-24130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780735#comment-16780735 ]
William Wong edited comment on SPARK-24130 at 3/28/19 7:31 AM:
---------------------------------------------------------------
Hi [~jliwork], Yes. It will be a very valuable enhancement. Appreciate if you can let us know the progress. Many thanks.
was (Author: william1104):
Hi [~smilegator], Yes. It will be a very valuable enhancement. Appreciate if you can let us know the progress. Many thanks.
> Data Source V2: Join Push Down
> ------------------------------
>
> Key: SPARK-24130
> URL: https://issues.apache.org/jira/browse/SPARK-24130
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 2.3.0
> Reporter: Jia Li
> Priority: Major
> Attachments: Data Source V2 Join Push Down.pdf
>
>
> Spark applications often directly query external data sources such as relational databases, or files. Spark provides Data Sources APIs for accessing structured data through Spark SQL. Data Sources APIs in both V1 and V2 support optimizations such as Filter push down and Column pruning which are subset of the functionality that can be pushed down to some data sources. We’re proposing to extend Data Sources APIs with join push down (JPD). Join push down significantly improves query performance by reducing the amount of data transfer and exploiting the capabilities of the data sources such as index access.
> Join push down design document is available [here|https://docs.google.com/document/d/1k-kRadTcUbxVfUQwqBbIXs_yPZMxh18-e-cz77O_TaE/edit?usp=sharing].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org