You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by James King <ja...@gmail.com> on 2015/03/18 10:38:05 UTC
Spark + Kafka
Hi All,
Which build of Spark is best when using Kafka?
Regards
jk
Re: Spark + Kafka
Posted by James King <ja...@gmail.com>.
Thanks Jeff, I'm planning to use it in standalone mode, OK will use hadoop
2.4 package. Chao!
On Wed, Mar 18, 2015 at 10:56 AM, Jeffrey Jedele <je...@gmail.com>
wrote:
> What you call "sub-category" are packages pre-built to run on certain
> Hadoop environments. It really depends on where you want to run Spark. As
> far as I know, this is mainly about the included HDFS binding - so if you
> just want to play around with Spark, any of the packages should be fine. I
> wouldn't use source though, because you'd have to compile it yourself.
>
> PS: Make sure to use "Reply to all". If you're not including the mailing
> list in the response, I'm the only one who will get your message.
>
> Regards,
> Jeff
>
> 2015-03-18 10:49 GMT+01:00 James King <ja...@gmail.com>:
>
>> Any sub-category recommendations hadoop, MapR, CDH?
>>
>> On Wed, Mar 18, 2015 at 10:48 AM, James King <ja...@gmail.com>
>> wrote:
>>
>>> Many thanks Jeff will give it a go.
>>>
>>> On Wed, Mar 18, 2015 at 10:47 AM, Jeffrey Jedele <
>>> jeffrey.jedele@gmail.com> wrote:
>>>
>>>> Probably 1.3.0 - it has some improvements in the included Kafka
>>>> receiver for streaming.
>>>>
>>>> https://spark.apache.org/releases/spark-release-1-3-0.html
>>>>
>>>> Regards,
>>>> Jeff
>>>>
>>>> 2015-03-18 10:38 GMT+01:00 James King <ja...@gmail.com>:
>>>>
>>>>> Hi All,
>>>>>
>>>>> Which build of Spark is best when using Kafka?
>>>>>
>>>>> Regards
>>>>> jk
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Spark + Kafka
Posted by Jeffrey Jedele <je...@gmail.com>.
What you call "sub-category" are packages pre-built to run on certain
Hadoop environments. It really depends on where you want to run Spark. As
far as I know, this is mainly about the included HDFS binding - so if you
just want to play around with Spark, any of the packages should be fine. I
wouldn't use source though, because you'd have to compile it yourself.
PS: Make sure to use "Reply to all". If you're not including the mailing
list in the response, I'm the only one who will get your message.
Regards,
Jeff
2015-03-18 10:49 GMT+01:00 James King <ja...@gmail.com>:
> Any sub-category recommendations hadoop, MapR, CDH?
>
> On Wed, Mar 18, 2015 at 10:48 AM, James King <ja...@gmail.com>
> wrote:
>
>> Many thanks Jeff will give it a go.
>>
>> On Wed, Mar 18, 2015 at 10:47 AM, Jeffrey Jedele <
>> jeffrey.jedele@gmail.com> wrote:
>>
>>> Probably 1.3.0 - it has some improvements in the included Kafka receiver
>>> for streaming.
>>>
>>> https://spark.apache.org/releases/spark-release-1-3-0.html
>>>
>>> Regards,
>>> Jeff
>>>
>>> 2015-03-18 10:38 GMT+01:00 James King <ja...@gmail.com>:
>>>
>>>> Hi All,
>>>>
>>>> Which build of Spark is best when using Kafka?
>>>>
>>>> Regards
>>>> jk
>>>>
>>>
>>>
>>
>
Re: Spark + Kafka
Posted by Jeffrey Jedele <je...@gmail.com>.
Probably 1.3.0 - it has some improvements in the included Kafka receiver
for streaming.
https://spark.apache.org/releases/spark-release-1-3-0.html
Regards,
Jeff
2015-03-18 10:38 GMT+01:00 James King <ja...@gmail.com>:
> Hi All,
>
> Which build of Spark is best when using Kafka?
>
> Regards
> jk
>
Re: Spark + Kafka
Posted by James King <ja...@gmail.com>.
Many thanks all for the good responses, appreciated.
On Thu, Mar 19, 2015 at 8:36 AM, James King <ja...@gmail.com> wrote:
> Thanks Khanderao.
>
> On Wed, Mar 18, 2015 at 7:18 PM, Khanderao Kand Gmail <
> khanderao.kand@gmail.com> wrote:
>
>> I have used various version of spark (1.0, 1.2.1) without any issues .
>> Though I have not significantly used kafka with 1.3.0 , a preliminary
>> testing revealed no issues .
>>
>> - khanderao
>>
>>
>>
>> > On Mar 18, 2015, at 2:38 AM, James King <ja...@gmail.com> wrote:
>> >
>> > Hi All,
>> >
>> > Which build of Spark is best when using Kafka?
>> >
>> > Regards
>> > jk
>>
>
>
Re: Spark + Kafka
Posted by James King <ja...@gmail.com>.
Thanks Khanderao.
On Wed, Mar 18, 2015 at 7:18 PM, Khanderao Kand Gmail <
khanderao.kand@gmail.com> wrote:
> I have used various version of spark (1.0, 1.2.1) without any issues .
> Though I have not significantly used kafka with 1.3.0 , a preliminary
> testing revealed no issues .
>
> - khanderao
>
>
>
> > On Mar 18, 2015, at 2:38 AM, James King <ja...@gmail.com> wrote:
> >
> > Hi All,
> >
> > Which build of Spark is best when using Kafka?
> >
> > Regards
> > jk
>
Re: Spark + Kafka
Posted by Khanderao Kand Gmail <kh...@gmail.com>.
I have used various version of spark (1.0, 1.2.1) without any issues . Though I have not significantly used kafka with 1.3.0 , a preliminary testing revealed no issues .
- khanderao
> On Mar 18, 2015, at 2:38 AM, James King <ja...@gmail.com> wrote:
>
> Hi All,
>
> Which build of Spark is best when using Kafka?
>
> Regards
> jk
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org