You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sandeep Pal (JIRA)" <ji...@apache.org> on 2015/07/23 19:58:06 UTC
[jira] [Updated] (SPARK-9282) Join on Spark DataFrame with multiple
columns
[ https://issues.apache.org/jira/browse/SPARK-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sandeep Pal updated SPARK-9282:
-------------------------------
Description:
Filter on dataframe does not work if we have more than one column inside the filter. Nonetheless, it works on an RDD.
Following is the example:
df1.show()
age coolid depid empname
23 7 1 sandeep
21 8 2 john
24 9 1 cena
45 12 3 bob
20 7 4 tanay
12 8 5 gaurav
df1.filter(df1.age > 21 and df1.age < 45).show(10)
23 7 1 sandeep
21 8 2 john <-------------
24 9 1 cena
20 7 4 tanay <-------------
12 8 5 gaurav <--------------
was:
Filter on dataframe does not work if we have more than one column inside the filter. Nonetheless, it works on an RDD.
Following is the example:
df1.show()
age coolid depid empname
23 7 1 sandeep
21 8 2 john
24 9 1 cena
45 12 3 bob
20 7 4 tanay
12 8 5 gaurav
df1.filter(df1.age > 21 and df1.age < 45).show(10)
23 7 1 sandeep
21 8 2 john
24 9 1 cena
20 7 4 tanay
12 8 5 gaurav
> Join on Spark DataFrame with multiple columns
> ---------------------------------------------
>
> Key: SPARK-9282
> URL: https://issues.apache.org/jira/browse/SPARK-9282
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, Spark Shell, SQL
> Affects Versions: 1.3.0
> Environment: CDH 5.0 on CentOS6
> Reporter: Sandeep Pal
>
> Filter on dataframe does not work if we have more than one column inside the filter. Nonetheless, it works on an RDD.
> Following is the example:
> df1.show()
> age coolid depid empname
> 23 7 1 sandeep
> 21 8 2 john
> 24 9 1 cena
> 45 12 3 bob
> 20 7 4 tanay
> 12 8 5 gaurav
> df1.filter(df1.age > 21 and df1.age < 45).show(10)
> 23 7 1 sandeep
> 21 8 2 john <-------------
> 24 9 1 cena
> 20 7 4 tanay <-------------
> 12 8 5 gaurav <--------------
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org