You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Yuheng Du <yu...@gmail.com> on 2015/03/05 21:16:43 UTC
Set up kafka cluster
Hi everyone,
I am trying to set up a kafka cluster consisting of three machines. I wanna
run a benchmarking program in them. Can anyone recommend a step by step
tutorial/instruction of how I can do it?
Thanks.
best,
Yuheng
Re: Set up kafka cluster
Posted by Yuheng Du <yu...@gmail.com>.
will do, Thanks!
On Thu, Mar 5, 2015 at 3:35 PM, Gwen Shapira <gs...@cloudera.com> wrote:
> Did you take a look at the quick-start guide?
> https://kafka.apache.org/082/quickstart.html
>
> It shows how to set up a single node, how to validate that its working
> and then how to set up multi-node cluster.
>
> Good luck!
>
> On Thu, Mar 5, 2015 at 12:30 PM, Yuheng Du <yu...@gmail.com>
> wrote:
> > Thank you Gwen,
> >
> > I also need the kafka cluster continue to provide message brokering
> service
> > to a Storm cluster after the benchmarking. I am fairly new to cluster
> > setups. So is there an instruction telling me how to set up the
> three-node
> > kafka cluster before running benchmarking? That would be really helpful.
> >
> > Thanks gain.
> >
> > best,
> > Yuheng
> >
> > On Thu, Mar 5, 2015 at 3:22 PM, Gwen Shapira <gs...@cloudera.com>
> wrote:
> >
> >> Jay Kreps has a gist with step by step instructions for reproducing
> >> the benchmarks used by LinkedIn:
> >> https://gist.github.com/jkreps/c7ddb4041ef62a900e6c
> >>
> >> And the blog with the results:
> >>
> >>
> https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines
> >>
> >> Gwen
> >>
> >> On Thu, Mar 5, 2015 at 12:16 PM, Yuheng Du <yu...@gmail.com>
> >> wrote:
> >> > Hi everyone,
> >> >
> >> > I am trying to set up a kafka cluster consisting of three machines. I
> >> wanna
> >> > run a benchmarking program in them. Can anyone recommend a step by
> step
> >> > tutorial/instruction of how I can do it?
> >> >
> >> > Thanks.
> >> >
> >> > best,
> >> > Yuheng
> >>
>
Re: Set up kafka cluster
Posted by Gwen Shapira <gs...@cloudera.com>.
Did you take a look at the quick-start guide?
https://kafka.apache.org/082/quickstart.html
It shows how to set up a single node, how to validate that its working
and then how to set up multi-node cluster.
Good luck!
On Thu, Mar 5, 2015 at 12:30 PM, Yuheng Du <yu...@gmail.com> wrote:
> Thank you Gwen,
>
> I also need the kafka cluster continue to provide message brokering service
> to a Storm cluster after the benchmarking. I am fairly new to cluster
> setups. So is there an instruction telling me how to set up the three-node
> kafka cluster before running benchmarking? That would be really helpful.
>
> Thanks gain.
>
> best,
> Yuheng
>
> On Thu, Mar 5, 2015 at 3:22 PM, Gwen Shapira <gs...@cloudera.com> wrote:
>
>> Jay Kreps has a gist with step by step instructions for reproducing
>> the benchmarks used by LinkedIn:
>> https://gist.github.com/jkreps/c7ddb4041ef62a900e6c
>>
>> And the blog with the results:
>>
>> https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines
>>
>> Gwen
>>
>> On Thu, Mar 5, 2015 at 12:16 PM, Yuheng Du <yu...@gmail.com>
>> wrote:
>> > Hi everyone,
>> >
>> > I am trying to set up a kafka cluster consisting of three machines. I
>> wanna
>> > run a benchmarking program in them. Can anyone recommend a step by step
>> > tutorial/instruction of how I can do it?
>> >
>> > Thanks.
>> >
>> > best,
>> > Yuheng
>>
Re: Set up kafka cluster
Posted by Yuheng Du <yu...@gmail.com>.
Thank you Gwen,
I also need the kafka cluster continue to provide message brokering service
to a Storm cluster after the benchmarking. I am fairly new to cluster
setups. So is there an instruction telling me how to set up the three-node
kafka cluster before running benchmarking? That would be really helpful.
Thanks gain.
best,
Yuheng
On Thu, Mar 5, 2015 at 3:22 PM, Gwen Shapira <gs...@cloudera.com> wrote:
> Jay Kreps has a gist with step by step instructions for reproducing
> the benchmarks used by LinkedIn:
> https://gist.github.com/jkreps/c7ddb4041ef62a900e6c
>
> And the blog with the results:
>
> https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines
>
> Gwen
>
> On Thu, Mar 5, 2015 at 12:16 PM, Yuheng Du <yu...@gmail.com>
> wrote:
> > Hi everyone,
> >
> > I am trying to set up a kafka cluster consisting of three machines. I
> wanna
> > run a benchmarking program in them. Can anyone recommend a step by step
> > tutorial/instruction of how I can do it?
> >
> > Thanks.
> >
> > best,
> > Yuheng
>
Re: Set up kafka cluster
Posted by Gwen Shapira <gs...@cloudera.com>.
Jay Kreps has a gist with step by step instructions for reproducing
the benchmarks used by LinkedIn:
https://gist.github.com/jkreps/c7ddb4041ef62a900e6c
And the blog with the results:
https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines
Gwen
On Thu, Mar 5, 2015 at 12:16 PM, Yuheng Du <yu...@gmail.com> wrote:
> Hi everyone,
>
> I am trying to set up a kafka cluster consisting of three machines. I wanna
> run a benchmarking program in them. Can anyone recommend a step by step
> tutorial/instruction of how I can do it?
>
> Thanks.
>
> best,
> Yuheng