You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Mich Talebzadeh <mi...@gmail.com> on 2016/04/17 11:04:47 UTC
Testing Kafka interface using Flink interactive shell
Hi,
IN Spark shell I can load Kafka jar file through spark-shell option --jar
spark-shell --master spark://50.140.197.217:7077 --jars
,/home/hduser/jars/spark-streaming-kafka-assembly_2.10-1.6.1.jar
This works fine.
In Flink I have added the jar file
/home/hduser/jars/flink-connector-kafka-0.10.1.jar to the CLASSPATH.
However I don't get any support for it within flink shell
Scala-Flink> import org.apache.flink.streaming.connectors.kafka
<console>:54: error: object connectors is not a member of package
org.apache.flink.streaming
import org.apache.flink.streaming.connectors.kafka
Any ideas will be appreciated
^
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
Re: Testing Kafka interface using Flink interactive shell
Posted by Mich Talebzadeh <mi...@gmail.com>.
Thanks Chiwan. It worked.
Now I have this simple streaming program in Spark Scala that gets streaming
data via Kafka. It is pretty simple. Please see attached.
I am trying to make it work with Flink + Kafka
Any hints will be appreciated.
Thanks
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
On 18 April 2016 at 02:43, Chiwan Park <ch...@apache.org> wrote:
> Hi Mich,
>
> You can add external dependencies to Scala shell using `--addclasspath`
> option. There is more detail description in documentation [1].
>
> [1]:
> https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/scala_shell.html#adding-external-dependencies
>
> Regards,
> Chiwan Park
>
> > On Apr 17, 2016, at 6:04 PM, Mich Talebzadeh <mi...@gmail.com>
> wrote:
> >
> > Hi,
> >
> > IN Spark shell I can load Kafka jar file through spark-shell option --jar
> >
> > spark-shell --master spark://50.140.197.217:7077 --jars
> ,/home/hduser/jars/spark-streaming-kafka-assembly_2.10-1.6.1.jar
> >
> > This works fine.
> >
> > In Flink I have added the jar file
> /home/hduser/jars/flink-connector-kafka-0.10.1.jar to the CLASSPATH.
> >
> > However I don't get any support for it within flink shell
> >
> > Scala-Flink> import org.apache.flink.streaming.connectors.kafka
> > <console>:54: error: object connectors is not a member of package
> org.apache.flink.streaming
> > import org.apache.flink.streaming.connectors.kafka
> >
> >
> > Any ideas will be appreciated
> > ^
> >
> > Dr Mich Talebzadeh
> >
> > LinkedIn
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> >
> > http://talebzadehmich.wordpress.com
> >
>
>
Re: Testing Kafka interface using Flink interactive shell
Posted by Chiwan Park <ch...@apache.org>.
Hi Mich,
You can add external dependencies to Scala shell using `--addclasspath` option. There is more detail description in documentation [1].
[1]: https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/scala_shell.html#adding-external-dependencies
Regards,
Chiwan Park
> On Apr 17, 2016, at 6:04 PM, Mich Talebzadeh <mi...@gmail.com> wrote:
>
> Hi,
>
> IN Spark shell I can load Kafka jar file through spark-shell option --jar
>
> spark-shell --master spark://50.140.197.217:7077 --jars ,/home/hduser/jars/spark-streaming-kafka-assembly_2.10-1.6.1.jar
>
> This works fine.
>
> In Flink I have added the jar file /home/hduser/jars/flink-connector-kafka-0.10.1.jar to the CLASSPATH.
>
> However I don't get any support for it within flink shell
>
> Scala-Flink> import org.apache.flink.streaming.connectors.kafka
> <console>:54: error: object connectors is not a member of package org.apache.flink.streaming
> import org.apache.flink.streaming.connectors.kafka
>
>
> Any ideas will be appreciated
> ^
>
> Dr Mich Talebzadeh
>
> LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>
> http://talebzadehmich.wordpress.com
>