You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by "Devi P.V" <de...@gmail.com> on 2016/10/16 14:51:35 UTC
Couchbase-Spark 2.0.0
Hi all,
I am trying to read data from couchbase using spark 2.0.0.I need to fetch
complete data from a bucket as Rdd.How can I solve this?Does spark 2.0.0
support couchbase?Please help.
Thanks
Re: Couchbase-Spark 2.0.0
Posted by Sean Owen <so...@cloudera.com>.
You're now asking about couchbase code, so this isn't the best place to
ask. Head to couchbase forums.
On Mon, Oct 17, 2016 at 10:14 AM Devi P.V <de...@gmail.com> wrote:
> Hi,
> I tried with the following code
>
> import com.couchbase.spark._
> val conf = new SparkConf()
> .setAppName(this.getClass.getName)
> .setMaster("local[*]")
> .set("com.couchbase.bucket.bucketName","password")
> .set("com.couchbase.nodes", "node")
> .set ("com.couchbase.queryEnabled", "true")
> val sc = new SparkContext(conf)
>
> I need full document from bucket,so i gave query like this,
>
> val query = "SELECT META(`bucketName`).id as id FROM `bucketName` "
> sc
> .couchbaseQuery(Query.simple(query))
> .map(_.value.getString("id"))
> .couchbaseGet[JsonDocument]()
> .collect()
> .foreach(println)
>
> But it can't take Query.simple(query)
>
> I used libraryDependencies += "com.couchbase.client" %
> "spark-connector_2.11" % "1.2.1" in built.sbt.
> Is my query wrong or anything else needed to import?
>
>
> Please help.
>
> On Sun, Oct 16, 2016 at 8:23 PM, Rodrick Brown <
> rodrick@orchardplatform.com> wrote:
>
>
>
> On Sun, Oct 16, 2016 at 10:51 AM, Devi P.V <de...@gmail.com> wrote:
>
> Hi all,
> I am trying to read data from couchbase using spark 2.0.0.I need to fetch
> complete data from a bucket as Rdd.How can I solve this?Does spark 2.0.0
> support couchbase?Please help.
>
> Thanks
>
> https://github.com/couchbase/couchbase-spark-connector
>
>
> --
>
> [image: Orchard Platform] <http://www.orchardplatform.com/>
>
> *Rodrick Brown */ *DevOPs*
>
> 9174456839 / rodrick@orchardplatform.com
>
> Orchard Platform
> 101 5th Avenue, 4th Floor, New York, NY
>
> *NOTICE TO RECIPIENTS*: This communication is confidential and intended
> for the use of the addressee only. If you are not an intended recipient of
> this communication, please delete it immediately and notify the sender by
> return email. Unauthorized reading, dissemination, distribution or copying
> of this communication is prohibited. This communication does not constitute
> an offer to sell or a solicitation of an indication of interest to purchase
> any loan, security or any other financial product or instrument, nor is it
> an offer to sell or a solicitation of an indication of interest to purchase
> any products or services to any persons who are prohibited from receiving
> such information under applicable law. The contents of this communication
> may not be accurate or complete and are subject to change without notice.
> As such, Orchard App, Inc. (including its subsidiaries and affiliates,
> "Orchard") makes no representation regarding the accuracy or completeness
> of the information contained herein. The intended recipient is advised to
> consult its own professional advisors, including those specializing in
> legal, tax and accounting matters. Orchard does not provide legal, tax or
> accounting advice.
>
>
>
Re: Couchbase-Spark 2.0.0
Posted by "Devi P.V" <de...@gmail.com>.
Hi,
I tried with the following code
import com.couchbase.spark._
val conf = new SparkConf()
.setAppName(this.getClass.getName)
.setMaster("local[*]")
.set("com.couchbase.bucket.bucketName","password")
.set("com.couchbase.nodes", "node")
.set ("com.couchbase.queryEnabled", "true")
val sc = new SparkContext(conf)
I need full document from bucket,so i gave query like this,
val query = "SELECT META(`bucketName`).id as id FROM `bucketName` "
sc
.couchbaseQuery(Query.simple(query))
.map(_.value.getString("id"))
.couchbaseGet[JsonDocument]()
.collect()
.foreach(println)
But it can't take Query.simple(query)
I used libraryDependencies += "com.couchbase.client" %
"spark-connector_2.11" % "1.2.1" in built.sbt.
Is my query wrong or anything else needed to import?
Please help.
On Sun, Oct 16, 2016 at 8:23 PM, Rodrick Brown <ro...@orchardplatform.com>
wrote:
>
>
> On Sun, Oct 16, 2016 at 10:51 AM, Devi P.V <de...@gmail.com> wrote:
>
>> Hi all,
>> I am trying to read data from couchbase using spark 2.0.0.I need to fetch
>> complete data from a bucket as Rdd.How can I solve this?Does spark 2.0.0
>> support couchbase?Please help.
>>
>> Thanks
>>
> https://github.com/couchbase/couchbase-spark-connector
>
>
> --
>
> [image: Orchard Platform] <http://www.orchardplatform.com/>
>
> *Rodrick Brown */ *DevOPs*
>
> 9174456839 / rodrick@orchardplatform.com
>
> Orchard Platform
> 101 5th Avenue, 4th Floor, New York, NY
>
> *NOTICE TO RECIPIENTS*: This communication is confidential and intended
> for the use of the addressee only. If you are not an intended recipient of
> this communication, please delete it immediately and notify the sender by
> return email. Unauthorized reading, dissemination, distribution or copying
> of this communication is prohibited. This communication does not constitute
> an offer to sell or a solicitation of an indication of interest to purchase
> any loan, security or any other financial product or instrument, nor is it
> an offer to sell or a solicitation of an indication of interest to purchase
> any products or services to any persons who are prohibited from receiving
> such information under applicable law. The contents of this communication
> may not be accurate or complete and are subject to change without notice.
> As such, Orchard App, Inc. (including its subsidiaries and affiliates,
> "Orchard") makes no representation regarding the accuracy or completeness
> of the information contained herein. The intended recipient is advised to
> consult its own professional advisors, including those specializing in
> legal, tax and accounting matters. Orchard does not provide legal, tax or
> accounting advice.
>
Re: Couchbase-Spark 2.0.0
Posted by Rodrick Brown <ro...@orchardplatform.com>.
On Sun, Oct 16, 2016 at 10:51 AM, Devi P.V <de...@gmail.com> wrote:
> Hi all,
> I am trying to read data from couchbase using spark 2.0.0.I need to fetch
> complete data from a bucket as Rdd.How can I solve this?Does spark 2.0.0
> support couchbase?Please help.
>
> Thanks
>
https://github.com/couchbase/couchbase-spark-connector
--
[image: Orchard Platform] <http://www.orchardplatform.com/>
*Rodrick Brown */ *DevOPs*
9174456839 / rodrick@orchardplatform.com
Orchard Platform
101 5th Avenue, 4th Floor, New York, NY
--
*NOTICE TO RECIPIENTS*: This communication is confidential and intended for
the use of the addressee only. If you are not an intended recipient of this
communication, please delete it immediately and notify the sender by return
email. Unauthorized reading, dissemination, distribution or copying of this
communication is prohibited. This communication does not constitute an
offer to sell or a solicitation of an indication of interest to purchase
any loan, security or any other financial product or instrument, nor is it
an offer to sell or a solicitation of an indication of interest to purchase
any products or services to any persons who are prohibited from receiving
such information under applicable law. The contents of this communication
may not be accurate or complete and are subject to change without notice.
As such, Orchard App, Inc. (including its subsidiaries and affiliates,
"Orchard") makes no representation regarding the accuracy or completeness
of the information contained herein. The intended recipient is advised to
consult its own professional advisors, including those specializing in
legal, tax and accounting matters. Orchard does not provide legal, tax or
accounting advice.