You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Maisnam Ns <ma...@gmail.com> on 2014/09/21 00:16:30 UTC

Spark streaming twitter exception

HI ,

Can somebody help me with adding library dependencies in  my build.sbt so
that the java.lang.NoClassDefFoundError issue can be resolved.

My sbt (only the dependencies part) ->

 libraryDependencies ++= Seq(
  "org.apache.spark" %% "spark-core" % "1.0.1" ,
  "org.apache.spark" %% "spark-streaming" % "1.0.1" ,
  "org.apache.spark" %% "spark-streaming-twitter" % "1.0.1"
)

The exception , I am getting  ->
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/spark/streaming/twitter/TwitterUtils$     at
TwitterApp$.main(TwitterApp.scala:64)
   at TwitterApp.main(TwitterApp.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:330)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

The command to submit the jar -> bin/spark-submit --class "TwitterApp"
--master local[4] twitterapp-project_2.10-1.0.jar

Thanks in advance.

Regards,
NIranjan

Re: Spark streaming twitter exception

Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Awesome! :]

Thanks
Best Regards

On Mon, Sep 22, 2014 at 6:10 PM, Maisnam Ns <ma...@gmail.com> wrote:

> Yes, it works with these dependencies . Thank you. Now , I am able to run
> Spark twitter streaming .
>
> On Mon, Sep 22, 2014 at 12:09 PM, Akhil Das <ak...@sigmoidanalytics.com>
> wrote:
>
>> Can you try adding these dependencies?
>>
>> libraryDependencies += "org.apache.spark" %
>> "spark-streaming-twitter_2.10" % "1.0.1"
>> libraryDependencies += "org.twitter4j" % "twitter4j-core" % "4.0.0"
>> libraryDependencies += "org.twitter4j" % "twitter4j" % "4.0.0"
>>
>>
>> And make sure these 3 jars are downloaded and present in the classpath.
>>
>> Thanks
>> Best Regards
>>
>> On Sun, Sep 21, 2014 at 3:46 AM, Maisnam Ns <ma...@gmail.com> wrote:
>>
>>> HI ,
>>>
>>> Can somebody help me with adding library dependencies in  my build.sbt
>>> so that the java.lang.NoClassDefFoundError issue can be resolved.
>>>
>>> My sbt (only the dependencies part) ->
>>>
>>>  libraryDependencies ++= Seq(
>>>   "org.apache.spark" %% "spark-core" % "1.0.1" ,
>>>   "org.apache.spark" %% "spark-streaming" % "1.0.1" ,
>>>   "org.apache.spark" %% "spark-streaming-twitter" % "1.0.1"
>>> )
>>>
>>> The exception , I am getting  ->
>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>> org/apache/spark/streaming/twitter/TwitterUtils$     at
>>> TwitterApp$.main(TwitterApp.scala:64)
>>>    at TwitterApp.main(TwitterApp.scala)
>>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>     at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>     at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>     at java.lang.reflect.Method.invoke(Method.java:606)
>>>     at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:330)
>>>     at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
>>>     at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>>>
>>> The command to submit the jar -> bin/spark-submit --class "TwitterApp"
>>> --master local[4] twitterapp-project_2.10-1.0.jar
>>>
>>> Thanks in advance.
>>>
>>> Regards,
>>> NIranjan
>>>
>>>
>>
>

Re: Spark streaming twitter exception

Posted by Maisnam Ns <ma...@gmail.com>.
Yes, it works with these dependencies . Thank you. Now , I am able to run
Spark twitter streaming .

On Mon, Sep 22, 2014 at 12:09 PM, Akhil Das <ak...@sigmoidanalytics.com>
wrote:

> Can you try adding these dependencies?
>
> libraryDependencies += "org.apache.spark" % "spark-streaming-twitter_2.10"
> % "1.0.1"
> libraryDependencies += "org.twitter4j" % "twitter4j-core" % "4.0.0"
> libraryDependencies += "org.twitter4j" % "twitter4j" % "4.0.0"
>
>
> And make sure these 3 jars are downloaded and present in the classpath.
>
> Thanks
> Best Regards
>
> On Sun, Sep 21, 2014 at 3:46 AM, Maisnam Ns <ma...@gmail.com> wrote:
>
>> HI ,
>>
>> Can somebody help me with adding library dependencies in  my build.sbt so
>> that the java.lang.NoClassDefFoundError issue can be resolved.
>>
>> My sbt (only the dependencies part) ->
>>
>>  libraryDependencies ++= Seq(
>>   "org.apache.spark" %% "spark-core" % "1.0.1" ,
>>   "org.apache.spark" %% "spark-streaming" % "1.0.1" ,
>>   "org.apache.spark" %% "spark-streaming-twitter" % "1.0.1"
>> )
>>
>> The exception , I am getting  ->
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/spark/streaming/twitter/TwitterUtils$     at
>> TwitterApp$.main(TwitterApp.scala:64)
>>    at TwitterApp.main(TwitterApp.scala)
>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>     at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>     at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>     at java.lang.reflect.Method.invoke(Method.java:606)
>>     at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:330)
>>     at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
>>     at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>>
>> The command to submit the jar -> bin/spark-submit --class "TwitterApp"
>> --master local[4] twitterapp-project_2.10-1.0.jar
>>
>> Thanks in advance.
>>
>> Regards,
>> NIranjan
>>
>>
>

Re: Spark streaming twitter exception

Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Can you try adding these dependencies?

libraryDependencies += "org.apache.spark" % "spark-streaming-twitter_2.10"
% "1.0.1"
libraryDependencies += "org.twitter4j" % "twitter4j-core" % "4.0.0"
libraryDependencies += "org.twitter4j" % "twitter4j" % "4.0.0"


And make sure these 3 jars are downloaded and present in the classpath.

Thanks
Best Regards

On Sun, Sep 21, 2014 at 3:46 AM, Maisnam Ns <ma...@gmail.com> wrote:

> HI ,
>
> Can somebody help me with adding library dependencies in  my build.sbt so
> that the java.lang.NoClassDefFoundError issue can be resolved.
>
> My sbt (only the dependencies part) ->
>
>  libraryDependencies ++= Seq(
>   "org.apache.spark" %% "spark-core" % "1.0.1" ,
>   "org.apache.spark" %% "spark-streaming" % "1.0.1" ,
>   "org.apache.spark" %% "spark-streaming-twitter" % "1.0.1"
> )
>
> The exception , I am getting  ->
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/spark/streaming/twitter/TwitterUtils$     at
> TwitterApp$.main(TwitterApp.scala:64)
>    at TwitterApp.main(TwitterApp.scala)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:330)
>     at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
>     at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>
> The command to submit the jar -> bin/spark-submit --class "TwitterApp"
> --master local[4] twitterapp-project_2.10-1.0.jar
>
> Thanks in advance.
>
> Regards,
> NIranjan
>
>