You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Maisnam Ns <ma...@gmail.com> on 2014/09/21 00:16:30 UTC
Spark streaming twitter exception
HI ,
Can somebody help me with adding library dependencies in my build.sbt so
that the java.lang.NoClassDefFoundError issue can be resolved.
My sbt (only the dependencies part) ->
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "1.0.1" ,
"org.apache.spark" %% "spark-streaming" % "1.0.1" ,
"org.apache.spark" %% "spark-streaming-twitter" % "1.0.1"
)
The exception , I am getting ->
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/spark/streaming/twitter/TwitterUtils$ at
TwitterApp$.main(TwitterApp.scala:64)
at TwitterApp.main(TwitterApp.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:330)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
The command to submit the jar -> bin/spark-submit --class "TwitterApp"
--master local[4] twitterapp-project_2.10-1.0.jar
Thanks in advance.
Regards,
NIranjan
Re: Spark streaming twitter exception
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Awesome! :]
Thanks
Best Regards
On Mon, Sep 22, 2014 at 6:10 PM, Maisnam Ns <ma...@gmail.com> wrote:
> Yes, it works with these dependencies . Thank you. Now , I am able to run
> Spark twitter streaming .
>
> On Mon, Sep 22, 2014 at 12:09 PM, Akhil Das <ak...@sigmoidanalytics.com>
> wrote:
>
>> Can you try adding these dependencies?
>>
>> libraryDependencies += "org.apache.spark" %
>> "spark-streaming-twitter_2.10" % "1.0.1"
>> libraryDependencies += "org.twitter4j" % "twitter4j-core" % "4.0.0"
>> libraryDependencies += "org.twitter4j" % "twitter4j" % "4.0.0"
>>
>>
>> And make sure these 3 jars are downloaded and present in the classpath.
>>
>> Thanks
>> Best Regards
>>
>> On Sun, Sep 21, 2014 at 3:46 AM, Maisnam Ns <ma...@gmail.com> wrote:
>>
>>> HI ,
>>>
>>> Can somebody help me with adding library dependencies in my build.sbt
>>> so that the java.lang.NoClassDefFoundError issue can be resolved.
>>>
>>> My sbt (only the dependencies part) ->
>>>
>>> libraryDependencies ++= Seq(
>>> "org.apache.spark" %% "spark-core" % "1.0.1" ,
>>> "org.apache.spark" %% "spark-streaming" % "1.0.1" ,
>>> "org.apache.spark" %% "spark-streaming-twitter" % "1.0.1"
>>> )
>>>
>>> The exception , I am getting ->
>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>> org/apache/spark/streaming/twitter/TwitterUtils$ at
>>> TwitterApp$.main(TwitterApp.scala:64)
>>> at TwitterApp.main(TwitterApp.scala)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>> at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:330)
>>> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
>>> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>>>
>>> The command to submit the jar -> bin/spark-submit --class "TwitterApp"
>>> --master local[4] twitterapp-project_2.10-1.0.jar
>>>
>>> Thanks in advance.
>>>
>>> Regards,
>>> NIranjan
>>>
>>>
>>
>
Re: Spark streaming twitter exception
Posted by Maisnam Ns <ma...@gmail.com>.
Yes, it works with these dependencies . Thank you. Now , I am able to run
Spark twitter streaming .
On Mon, Sep 22, 2014 at 12:09 PM, Akhil Das <ak...@sigmoidanalytics.com>
wrote:
> Can you try adding these dependencies?
>
> libraryDependencies += "org.apache.spark" % "spark-streaming-twitter_2.10"
> % "1.0.1"
> libraryDependencies += "org.twitter4j" % "twitter4j-core" % "4.0.0"
> libraryDependencies += "org.twitter4j" % "twitter4j" % "4.0.0"
>
>
> And make sure these 3 jars are downloaded and present in the classpath.
>
> Thanks
> Best Regards
>
> On Sun, Sep 21, 2014 at 3:46 AM, Maisnam Ns <ma...@gmail.com> wrote:
>
>> HI ,
>>
>> Can somebody help me with adding library dependencies in my build.sbt so
>> that the java.lang.NoClassDefFoundError issue can be resolved.
>>
>> My sbt (only the dependencies part) ->
>>
>> libraryDependencies ++= Seq(
>> "org.apache.spark" %% "spark-core" % "1.0.1" ,
>> "org.apache.spark" %% "spark-streaming" % "1.0.1" ,
>> "org.apache.spark" %% "spark-streaming-twitter" % "1.0.1"
>> )
>>
>> The exception , I am getting ->
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/spark/streaming/twitter/TwitterUtils$ at
>> TwitterApp$.main(TwitterApp.scala:64)
>> at TwitterApp.main(TwitterApp.scala)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:330)
>> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
>> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>>
>> The command to submit the jar -> bin/spark-submit --class "TwitterApp"
>> --master local[4] twitterapp-project_2.10-1.0.jar
>>
>> Thanks in advance.
>>
>> Regards,
>> NIranjan
>>
>>
>
Re: Spark streaming twitter exception
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Can you try adding these dependencies?
libraryDependencies += "org.apache.spark" % "spark-streaming-twitter_2.10"
% "1.0.1"
libraryDependencies += "org.twitter4j" % "twitter4j-core" % "4.0.0"
libraryDependencies += "org.twitter4j" % "twitter4j" % "4.0.0"
And make sure these 3 jars are downloaded and present in the classpath.
Thanks
Best Regards
On Sun, Sep 21, 2014 at 3:46 AM, Maisnam Ns <ma...@gmail.com> wrote:
> HI ,
>
> Can somebody help me with adding library dependencies in my build.sbt so
> that the java.lang.NoClassDefFoundError issue can be resolved.
>
> My sbt (only the dependencies part) ->
>
> libraryDependencies ++= Seq(
> "org.apache.spark" %% "spark-core" % "1.0.1" ,
> "org.apache.spark" %% "spark-streaming" % "1.0.1" ,
> "org.apache.spark" %% "spark-streaming-twitter" % "1.0.1"
> )
>
> The exception , I am getting ->
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/spark/streaming/twitter/TwitterUtils$ at
> TwitterApp$.main(TwitterApp.scala:64)
> at TwitterApp.main(TwitterApp.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:330)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>
> The command to submit the jar -> bin/spark-submit --class "TwitterApp"
> --master local[4] twitterapp-project_2.10-1.0.jar
>
> Thanks in advance.
>
> Regards,
> NIranjan
>
>