You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by "Mail.com" <pr...@mail.com> on 2016/05/26 02:41:13 UTC
Kafka connection logs in Spark
Hi All,
I am connecting Spark 1.6 streaming to Kafka 0.8.2 with Kerberos. I ran spark streaming in debug mode, but do not see any log saying it connected to Kafka or topic etc. How could I enable that.
My spark streaming job runs but no messages are fetched from the RDD. Please suggest.
Thanks,
Pradeep
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Kafka connection logs in Spark
Posted by Cody Koeninger <co...@koeninger.org>.
Sounds like you better talk to Horton Works then
On Thu, May 26, 2016 at 2:33 PM, Mail.com <pr...@mail.com> wrote:
> Hi Cody,
>
> I used Horton Works jars for spark streaming that would enable get messages from Kafka with kerberos.
>
> Thanks,
> Pradeep
>
>
>> On May 26, 2016, at 11:04 AM, Cody Koeninger <co...@koeninger.org> wrote:
>>
>> I wouldn't expect kerberos to work with anything earlier than the beta
>> consumer for kafka 0.10
>>
>>> On Wed, May 25, 2016 at 9:41 PM, Mail.com <pr...@mail.com> wrote:
>>> Hi All,
>>>
>>> I am connecting Spark 1.6 streaming to Kafka 0.8.2 with Kerberos. I ran spark streaming in debug mode, but do not see any log saying it connected to Kafka or topic etc. How could I enable that.
>>>
>>> My spark streaming job runs but no messages are fetched from the RDD. Please suggest.
>>>
>>> Thanks,
>>> Pradeep
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: user-help@spark.apache.org
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Kafka connection logs in Spark
Posted by "Mail.com" <pr...@mail.com>.
Hi Cody,
I used Horton Works jars for spark streaming that would enable get messages from Kafka with kerberos.
Thanks,
Pradeep
> On May 26, 2016, at 11:04 AM, Cody Koeninger <co...@koeninger.org> wrote:
>
> I wouldn't expect kerberos to work with anything earlier than the beta
> consumer for kafka 0.10
>
>> On Wed, May 25, 2016 at 9:41 PM, Mail.com <pr...@mail.com> wrote:
>> Hi All,
>>
>> I am connecting Spark 1.6 streaming to Kafka 0.8.2 with Kerberos. I ran spark streaming in debug mode, but do not see any log saying it connected to Kafka or topic etc. How could I enable that.
>>
>> My spark streaming job runs but no messages are fetched from the RDD. Please suggest.
>>
>> Thanks,
>> Pradeep
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Kafka connection logs in Spark
Posted by Cody Koeninger <co...@koeninger.org>.
I wouldn't expect kerberos to work with anything earlier than the beta
consumer for kafka 0.10
On Wed, May 25, 2016 at 9:41 PM, Mail.com <pr...@mail.com> wrote:
> Hi All,
>
> I am connecting Spark 1.6 streaming to Kafka 0.8.2 with Kerberos. I ran spark streaming in debug mode, but do not see any log saying it connected to Kafka or topic etc. How could I enable that.
>
> My spark streaming job runs but no messages are fetched from the RDD. Please suggest.
>
> Thanks,
> Pradeep
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org