You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Harold Nguyen <ha...@nexgate.com> on 2014/10/29 17:22:58 UTC
Spark Streaming with Kinesis
Hi all,
I followed the guide here:
http://spark.apache.org/docs/latest/streaming-kinesis-integration.html
But got this error:
Exception in thread "main" java.lang.NoClassDefFoundError:
com/amazonaws/auth/AWSCredentialsProvider
Would you happen to know what dependency or jar is needed ?
Harold
Re: Spark Streaming with Kinesis
Posted by Matt Chu <mc...@kabam.com>.
I haven't tried this myself yet, but this sounds relevant:
https://github.com/apache/spark/pull/2535
Will be giving this a try today or so, will report back.
On Wednesday, October 29, 2014, Harold Nguyen <ha...@nexgate.com> wrote:
> Hi again,
>
> After getting through several dependencies, I finally got to this
> non-dependency type error:
>
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.apache.http.impl.conn.DefaultClientConnectionOperator.<init>(Lorg/apache/http/conn/scheme/SchemeRegistry;Lorg/apache/http/conn/DnsResolver;)V
>
> It look every similar to this post:
>
>
> http://stackoverflow.com/questions/24788949/nosuchmethoderror-while-running-aws-s3-client-on-spark-while-javap-shows-otherwi
>
> Since I'm a little new to everything, would someone be able to provide a
> step-by-step guidance for that ?
>
> Harold
>
> On Wed, Oct 29, 2014 at 9:22 AM, Harold Nguyen <harold@nexgate.com
> <javascript:_e(%7B%7D,'cvml','harold@nexgate.com');>> wrote:
>
>> Hi all,
>>
>> I followed the guide here:
>> http://spark.apache.org/docs/latest/streaming-kinesis-integration.html
>>
>> But got this error:
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> com/amazonaws/auth/AWSCredentialsProvider
>>
>> Would you happen to know what dependency or jar is needed ?
>>
>> Harold
>>
>>
>
Re: Spark Streaming with Kinesis
Posted by Harold Nguyen <ha...@nexgate.com>.
Hi again,
After getting through several dependencies, I finally got to this
non-dependency type error:
Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.http.impl.conn.DefaultClientConnectionOperator.<init>(Lorg/apache/http/conn/scheme/SchemeRegistry;Lorg/apache/http/conn/DnsResolver;)V
It look every similar to this post:
http://stackoverflow.com/questions/24788949/nosuchmethoderror-while-running-aws-s3-client-on-spark-while-javap-shows-otherwi
Since I'm a little new to everything, would someone be able to provide a
step-by-step guidance for that ?
Harold
On Wed, Oct 29, 2014 at 9:22 AM, Harold Nguyen <ha...@nexgate.com> wrote:
> Hi all,
>
> I followed the guide here:
> http://spark.apache.org/docs/latest/streaming-kinesis-integration.html
>
> But got this error:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> com/amazonaws/auth/AWSCredentialsProvider
>
> Would you happen to know what dependency or jar is needed ?
>
> Harold
>
>