You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by saucam <gi...@git.apache.org> on 2014/12/05 10:07:55 UTC
[GitHub] spark pull request: SPARK-4762: Add support for tuples in 'where i...
GitHub user saucam opened a pull request:
https://github.com/apache/spark/pull/3618
SPARK-4762: Add support for tuples in 'where in' clause query
Currently, in the where in clause the filter is applied only on a single column. We can enhance it to accept filter on multiple columns.
So current support is for queries like :
Select * from table where c1 in (value1,value2,...value n);
This added support for queries like :
Select * from table where (c1,c2,... cn) in ((value1,value2...value n), (value1' , value2' ... ,value n').... )
Also, added optimized version of where in clause of tuples , where we create a hashset of the filter tuples for matching rows.
This also requires a change in the hive parser since currently there is no support for multiple columns in IN clause.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/saucam/spark tuple_where_clause
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3618.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3618
----
commit c877926c64c7c6f2048d31759f35446c9cec1cdc
Author: Yash Datta <ya...@guavus.com>
Date: 2014-12-05T08:55:29Z
SPARK-4762: 1. Add support for tuples in 'where in' clause query
2. Also adds optimized version of the same, which uses hashset to filter rows
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-4762: Add support for tuples in 'where i...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3618#issuecomment-65764171
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-4762: Add support for tuples in 'where i...
Posted by saucam <gi...@git.apache.org>.
Github user saucam commented on the pull request:
https://github.com/apache/spark/pull/3618#issuecomment-65764552
@pwendell this PR requires a change in the hive parser for which i created a PR against hive trunk here : https://github.com/apache/hive/pull/25
can you please suggest if I need to open this request against some other branch which is used for spark build ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-4762: Add support for tuples in 'where i...
Posted by sryza <gi...@git.apache.org>.
Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/3618#issuecomment-65767357
@saucam mind tagging this PR as [SQL]?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4762][SQL]: Add support for tuples in '...
Posted by marmbrus <gi...@git.apache.org>.
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/3618#issuecomment-67244672
Thanks for working on this, but I think we want to avoid diverging from the standard hive parser. Otherwise upgrading hive versions could become very difficult. For that reason I suggest we close this issue and reopen it after this change is accepted to upstream Hive.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4762][SQL]: Add support for tuples in '...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/3618
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org