You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Kyle Prifogle (JIRA)" <ji...@apache.org> on 2018/07/18 17:41:00 UTC
[jira] [Commented] (SPARK-12126) JDBC datasource processes filters
only commonly pushed down.
[ https://issues.apache.org/jira/browse/SPARK-12126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16548135#comment-16548135 ]
Kyle Prifogle commented on SPARK-12126:
---------------------------------------
Whats the hold up on this? I've noticed that the PR has been closed.
In the case of pushing down `limit` it seems fairly straightforward to modify the query to append a limit before executing it.
> JDBC datasource processes filters only commonly pushed down.
> ------------------------------------------------------------
>
> Key: SPARK-12126
> URL: https://issues.apache.org/jira/browse/SPARK-12126
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Hyukjin Kwon
> Priority: Major
>
> As suggested [here|https://issues.apache.org/jira/browse/SPARK-9182?focusedCommentId=14955646&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14955646], Currently JDBC datasource only processes the filters pushed down from {{DataSourceStrategy}}.
> Unlike ORC or Parquet, this can process pretty a lot of filters (for example, a + b > 3) since it is just about string parsing.
> As [here|https://issues.apache.org/jira/browse/SPARK-9182?focusedCommentId=15031526&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15031526], using {{CatalystScan}} trait might be one of solutions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org