You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Gaurav1809 <ga...@gmail.com> on 2017/03/24 03:47:11 UTC

How to load "kafka" as a data source

Hi All,

I am running a simple command on spark-shell - like this. It's a piece of
structured streaming.

val lines = (spark
   .readStream
   .format("kafka")
   .option("kafka.bootstrap.servers", "localhost:9092")
   .option("subscribe", "test")
   .load()
   .selectExpr("CAST(value AS STRING)")
   .as[String]
    )

I also downloaded spark-streaming-kafka-0-10-assembly_2.11-2.1.0.jar and
placed it in jars folder. re ran the spark-shell and executed above command.
but no luck. getting following error.

java.lang.ClassNotFoundException: Failed to find data source: kafka. Please
find packages at http://spark.apache.org/third-party-projects.html

Please suggest if I am missing anything here. I am running spark on windows.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-load-kafka-as-a-data-source-tp28534.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org

Re: How to load "kafka" as a data source

Posted by Deepu Raj <de...@outlook.com>.
Please check tools ver are same throughout.

Thanks
Deepu

On Fri, 24 Mar 2017 14:47:11 +1100, Gaurav1809 <ga...@gmail.com>  
wrote:

> Hi All,
>
> I am running a simple command on spark-shell - like this. It's a piece of
> structured streaming.
>
> val lines = (spark
>    .readStream
>    .format("kafka")
>    .option("kafka.bootstrap.servers", "localhost:9092")
>    .option("subscribe", "test")
>    .load()
>    .selectExpr("CAST(value AS STRING)")
>    .as[String]
>     )
>
> I also downloaded spark-streaming-kafka-0-10-assembly_2.11-2.1.0.jar and
> placed it in jars folder. re ran the spark-shell and executed above  
> command.
> but no luck. getting following error.
>
> java.lang.ClassNotFoundException: Failed to find data source: kafka.  
> Please
> find packages at http://spark.apache.org/third-party-projects.html
>
> Please suggest if I am missing anything here. I am running spark on  
> windows.
>
>
>
> --
> View this message in context:  
> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-load-kafka-as-a-data-source-tp28534.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>


-- 
Using Opera's mail client: http://www.opera.com/mail/

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org