You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Abhinav Chowdary <ab...@gmail.com> on 2014/11/02 06:33:38 UTC
Re: Spark SQL : how to find element where a field is in a given
set
i figured it out it was just a missing import Thanks
On November 1, 2014 at 19:51:41 PDT, abhinav chowdary <ab...@gmail.com> wrote:I have same requirement of passing list of values to in clause, when i am trying to do i am getting below error scala> val longList = Seq[Expression]("a", "b") <console>:11: error: type mismatch; found : String("a") required: org.apache.spark.sql.catalyst.expressions.Expression val longList = Seq[Expression]("a", "b") Thanks On Fri, Aug 29, 2014 at 3:52 PM, Michael Armbrust <mi...@databricks.com> wrote:This feature was not part of that version. It will be in 1.1. On Fri, Aug 29, 2014 at 12:33 PM, Jaonary Rabarisoa <ja...@gmail.com> wrote:1.0.2 On Friday, August 29, 2014, Michael Armbrust <mi...@databricks.com> wrote: What version are you using? On Fri, Aug 29, 2014 at 2:22 AM, Jaonary Rabarisoa <ja...@gmail.com> wrote:Still not working for me. I got a compilation error : value in is not a member of Symbol. Any ideas ? On Fri, Aug 29, 2014 at 9:46 AM, Michael Armbrust <mi...@databricks.com> wrote:To pass a list to a variadic function you can use the type ascription :_* For example: val longList = Seq[Expression]("a", "b", ...) table("src").where('key in (longList: _*)) Also, note that I had to explicitly specify Expression as the type parameter of Seq to ensure that the compiler converts "a" and "b" into Spark SQL expressions. On Thu, Aug 28, 2014 at 11:52 PM, Jaonary Rabarisoa <ja...@gmail.com> wrote:ok, but what if I have a long list do I need to hard code like this every element of my list of is there a function that translate a list into a tuple ? On Fri, Aug 29, 2014 at 3:24 AM, Michael Armbrust <mi...@databricks.com> wrote: You don't need the Seq, as in is a variadic function. personTable.where('name in ("foo", "bar")) On Thu, Aug 28, 2014 at 3:09 AM, Jaonary Rabarisoa <ja...@gmail.com> wrote:Hi all, What is the expression that I should use with spark sql DSL if I need to retreive data with a field in a given set. For example : I have the following schema case class Person(name: String, age: Int) And I need to do something like : personTable.where('name in Seq("foo", "bar")) ? Cheers. Jaonary -- Warm RegardsAbhinav Chowdary
Re: Spark SQL : how to find element where a field is in a given set
Posted by Vikas Agarwal <vi...@infoobjects.com>.
Wrong mailing list.
On Sun, Nov 2, 2014 at 11:03 AM, Abhinav Chowdary <
abhinav.chowdary@gmail.com> wrote:
>
> i figured it out it was just a missing import
>
> Thanks
>
> On November 1, 2014 at 19:51:41 PDT, abhinav chowdary <
> abhinav.chowdary@gmail.com> wrote:
>
> I have same requirement of passing list of values to in clause, when i am
> trying to do
>
> i am getting below error
>
> scala> val longList = Seq[Expression]("a", "b")
> <console>:11: error: type mismatch;
> found : String("a")
> required: org.apache.spark.sql.catalyst.expressions.Expression
> val longList = Seq[Expression]("a", "b")
>
> Thanks
>
>
> On Fri, Aug 29, 2014 at 3:52 PM, Michael Armbrust <mi...@databricks.com>
> wrote:
>
> This feature was not part of that version. It will be in 1.1.
>
>
> On Fri, Aug 29, 2014 at 12:33 PM, Jaonary Rabarisoa <ja...@gmail.com>
> wrote:
>
>
> 1.0.2
>
>
> On Friday, August 29, 2014, Michael Armbrust <mi...@databricks.com>
> wrote:
>
> What version are you using?
>
>
>
> On Fri, Aug 29, 2014 at 2:22 AM, Jaonary Rabarisoa <ja...@gmail.com>
> wrote:
>
> Still not working for me. I got a compilation error : *value in is not a
> member of Symbol.* Any ideas ?
>
>
> On Fri, Aug 29, 2014 at 9:46 AM, Michael Armbrust <mi...@databricks.com>
> wrote:
>
> To pass a list to a variadic function you can use the type ascription :_*
>
> For example:
>
> val longList = Seq[Expression]("a", "b", ...)
> table("src").where('key in (longList: _*))
>
> Also, note that I had to explicitly specify Expression as the type
> parameter of Seq to ensure that the compiler converts "a" and "b" into
> Spark SQL expressions.
>
>
>
>
> On Thu, Aug 28, 2014 at 11:52 PM, Jaonary Rabarisoa <ja...@gmail.com>
> wrote:
>
> ok, but what if I have a long list do I need to hard code like this every
> element of my list of is there a function that translate a list into a
> tuple ?
>
>
> On Fri, Aug 29, 2014 at 3:24 AM, Michael Armbrust <mi...@databricks.com>
> wrote:
>
> You don't need the Seq, as in is a variadic function.
>
> personTable.where('name in ("foo", "bar"))
>
>
>
> On Thu, Aug 28, 2014 at 3:09 AM, Jaonary Rabarisoa <ja...@gmail.com>
> wrote:
>
> Hi all,
>
> What is the expression that I should use with spark sql DSL if I need to
> retreive
> data with a field in a given set.
> For example :
>
> I have the following schema
>
> case class Person(name: String, age: Int)
>
> And I need to do something like :
>
> personTable.where('name in Seq("foo", "bar")) ?
>
>
> Cheers.
>
>
> Jaonary
>
>
>
>
>
>
>
>
>
>
>
>
> --
> Warm Regards
> Abhinav Chowdary
>
>
--
Regards,
Vikas Agarwal
91 – 9928301411
InfoObjects, Inc.
Execution Matters
http://www.infoobjects.com
2041 Mission College Boulevard, #280
Santa Clara, CA 95054
+1 (408) 988-2000 Work
+1 (408) 716-2726 Fax