You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by huaxingao <gi...@git.apache.org> on 2015/12/17 06:39:42 UTC
[GitHub] spark pull request: [SPARK-12387][SQL]JDBC IN operator push down
GitHub user huaxingao opened a pull request:
https://github.com/apache/spark/pull/10345
[SPARK-12387][SQL]JDBC IN operator push down
Will push down SQL IN operator such as the following to JDBC datasource
SELECT column_name(s)
FROM table_name
WHERE column_name IN (value1,value2,...)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/huaxingao/spark spark-12387
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/10345.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #10345
----
commit bb83a9ca666184e9a6f246f295d20f279bbd8a79
Author: Huaxin Gao <hu...@oc0558782468.ibm.com>
Date: 2015-12-17T05:32:07Z
[SPARK-12387][SQL]JDBC IN operator push down
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12387][SQL]JDBC IN operator push down
Posted by bomeng <gi...@git.apache.org>.
Github user bomeng commented on a diff in the pull request:
https://github.com/apache/spark/pull/10345#discussion_r47951480
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala ---
@@ -269,6 +269,13 @@ private[sql] class JDBCRDD(
case stringValue: String => s"'${escapeSql(stringValue)}'"
case timestampValue: Timestamp => "'" + timestampValue + "'"
case dateValue: Date => "'" + dateValue + "'"
+ case objectValue: Array[Object] => {
+ val str = objectValue.map {
+ case string: String => "'" + string + "'"
+ case other => other.toString
--- End diff --
case other => s"'${escapeSql(other)}'"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12387][SQL]JDBC IN operator push down
Posted by bomeng <gi...@git.apache.org>.
Github user bomeng commented on a diff in the pull request:
https://github.com/apache/spark/pull/10345#discussion_r47951376
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala ---
@@ -269,6 +269,13 @@ private[sql] class JDBCRDD(
case stringValue: String => s"'${escapeSql(stringValue)}'"
case timestampValue: Timestamp => "'" + timestampValue + "'"
case dateValue: Date => "'" + dateValue + "'"
+ case objectValue: Array[Object] => {
+ val str = objectValue.map {
+ case string: String => "'" + string + "'"
--- End diff --
change line 274 to:
case stringValue: String => s"'${escapeSql(string)}'"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12387][SQL]JDBC IN operator push down
Posted by huaxingao <gi...@git.apache.org>.
Github user huaxingao commented on the pull request:
https://github.com/apache/spark/pull/10345#issuecomment-165849528
Close for now. Will put the filter changes in one PR so it's easier to merge
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12387][SQL]JDBC IN operator push down
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10345#issuecomment-165344249
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12387][SQL]JDBC IN operator push down
Posted by huaxingao <gi...@git.apache.org>.
Github user huaxingao commented on a diff in the pull request:
https://github.com/apache/spark/pull/10345#discussion_r47965769
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala ---
@@ -269,6 +269,13 @@ private[sql] class JDBCRDD(
case stringValue: String => s"'${escapeSql(stringValue)}'"
case timestampValue: Timestamp => "'" + timestampValue + "'"
case dateValue: Date => "'" + dateValue + "'"
+ case objectValue: Array[Object] => {
+ val str = objectValue.map {
+ case string: String => "'" + string + "'"
--- End diff --
@bomeng
Thanks a lot for your comment. I will change.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12387][SQL]JDBC IN operator push down
Posted by huaxingao <gi...@git.apache.org>.
Github user huaxingao closed the pull request at:
https://github.com/apache/spark/pull/10345
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org