You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Chris Curtin <cu...@gmail.com> on 2013/01/16 16:26:40 UTC
Atlanta AJUG presentation on Kafka March 19th
Hi,
I'm going to be presenting an introduction to Kafka to the Atlanta Java
User's Group on March 19th.
http://www.meetup.com/atlantajug/events/99878712/
This will be a 'hey, this is a cool technology to check out' presentation
vs. a use case (since my day-job isn't using it in production yet). Anyone
have a good architecture overview pictures can use (with attribution of
course)?
I think I've watched every video presentation about Kafka I can find, but
if any of you who have presented want to share some common questions they
got or things they wish they'd included in their presentations please let
me know.
Thanks,
Chris
Re: Atlanta AJUG presentation on Kafka March 19th
Posted by Jun Rao <ju...@gmail.com>.
Chris,
You can find some numbers for LinkedIn usage in
http://sites.computer.org/debull/A12june/pipeline.pdf
More details about cross DC mirroring can be found in the following wiki.
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+mirroring+%28MirrorMaker%29
Thanks,
Jun
On Wed, Feb 13, 2013 at 6:22 AM, Chris Curtin <cu...@gmail.com>wrote:
> Thanks Jun & Jay,
>
> I did my first run through yesterday internally and a couple of the
> questions that came up were around performance, typical clusters and
> datacenter replication.
>
> I found some slides from 2011 and early 2012 about what LinkedIn's clusters
> consist of and some performance numbers. Are there some updated performance
> #s for a cluster configuration publicly available (#of machines, cores,
> disks etc. to # of messages written, consumed, # of consumers etc.)
>
> I found an old slide about datacenter replication before 0.8.0 Anyone have
> a description of how to do it with the new replication features?
>
> Thanks,
>
> Chris
>
>
> On Wed, Jan 16, 2013 at 12:35 PM, Jay Kreps <ja...@gmail.com> wrote:
>
> > Also, this wiki has a pretty good collection of presentations which may
> > give you ideas. If you want the source ppt or omnigraffle for any of the
> > presentations we made let us know.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+papers+and+presentations
> >
> > -Jay
> >
> >
> > On Wed, Jan 16, 2013 at 7:26 AM, Chris Curtin <curtin.chris@gmail.com
> > >wrote:
> >
> > > Hi,
> > >
> > > I'm going to be presenting an introduction to Kafka to the Atlanta Java
> > > User's Group on March 19th.
> > > http://www.meetup.com/atlantajug/events/99878712/
> > >
> > > This will be a 'hey, this is a cool technology to check out'
> presentation
> > > vs. a use case (since my day-job isn't using it in production yet).
> > Anyone
> > > have a good architecture overview pictures can use (with attribution of
> > > course)?
> > >
> > > I think I've watched every video presentation about Kafka I can find,
> but
> > > if any of you who have presented want to share some common questions
> they
> > > got or things they wish they'd included in their presentations please
> let
> > > me know.
> > >
> > > Thanks,
> > >
> > > Chris
> > >
> >
>
Re: Atlanta AJUG presentation on Kafka March 19th
Posted by Jay Kreps <ja...@gmail.com>.
I think the important thing to understand is that there are two
meanings for the word replication.
1. Mirroring is copying data from one cluster and writing it to
another. We use this to make sure that all client interaction is
always with a local cluster.
2. Intra-cluster replication is the feature in 0.8 and means we keep
an exact byte-for-byte replica of each partition within the cluster
with transparent failover whenever a machine dies.
Hope that clarifies things.
-Jay
On Wed, Feb 13, 2013 at 6:22 AM, Chris Curtin <cu...@gmail.com> wrote:
> Thanks Jun & Jay,
>
> I did my first run through yesterday internally and a couple of the
> questions that came up were around performance, typical clusters and
> datacenter replication.
>
> I found some slides from 2011 and early 2012 about what LinkedIn's clusters
> consist of and some performance numbers. Are there some updated performance
> #s for a cluster configuration publicly available (#of machines, cores,
> disks etc. to # of messages written, consumed, # of consumers etc.)
>
> I found an old slide about datacenter replication before 0.8.0 Anyone have
> a description of how to do it with the new replication features?
>
> Thanks,
>
> Chris
>
>
> On Wed, Jan 16, 2013 at 12:35 PM, Jay Kreps <ja...@gmail.com> wrote:
>
>> Also, this wiki has a pretty good collection of presentations which may
>> give you ideas. If you want the source ppt or omnigraffle for any of the
>> presentations we made let us know.
>>
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+papers+and+presentations
>>
>> -Jay
>>
>>
>> On Wed, Jan 16, 2013 at 7:26 AM, Chris Curtin <curtin.chris@gmail.com
>> >wrote:
>>
>> > Hi,
>> >
>> > I'm going to be presenting an introduction to Kafka to the Atlanta Java
>> > User's Group on March 19th.
>> > http://www.meetup.com/atlantajug/events/99878712/
>> >
>> > This will be a 'hey, this is a cool technology to check out' presentation
>> > vs. a use case (since my day-job isn't using it in production yet).
>> Anyone
>> > have a good architecture overview pictures can use (with attribution of
>> > course)?
>> >
>> > I think I've watched every video presentation about Kafka I can find, but
>> > if any of you who have presented want to share some common questions they
>> > got or things they wish they'd included in their presentations please let
>> > me know.
>> >
>> > Thanks,
>> >
>> > Chris
>> >
>>
Re: Atlanta AJUG presentation on Kafka March 19th
Posted by Chris Curtin <cu...@gmail.com>.
Thanks Jun & Jay,
I did my first run through yesterday internally and a couple of the
questions that came up were around performance, typical clusters and
datacenter replication.
I found some slides from 2011 and early 2012 about what LinkedIn's clusters
consist of and some performance numbers. Are there some updated performance
#s for a cluster configuration publicly available (#of machines, cores,
disks etc. to # of messages written, consumed, # of consumers etc.)
I found an old slide about datacenter replication before 0.8.0 Anyone have
a description of how to do it with the new replication features?
Thanks,
Chris
On Wed, Jan 16, 2013 at 12:35 PM, Jay Kreps <ja...@gmail.com> wrote:
> Also, this wiki has a pretty good collection of presentations which may
> give you ideas. If you want the source ppt or omnigraffle for any of the
> presentations we made let us know.
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+papers+and+presentations
>
> -Jay
>
>
> On Wed, Jan 16, 2013 at 7:26 AM, Chris Curtin <curtin.chris@gmail.com
> >wrote:
>
> > Hi,
> >
> > I'm going to be presenting an introduction to Kafka to the Atlanta Java
> > User's Group on March 19th.
> > http://www.meetup.com/atlantajug/events/99878712/
> >
> > This will be a 'hey, this is a cool technology to check out' presentation
> > vs. a use case (since my day-job isn't using it in production yet).
> Anyone
> > have a good architecture overview pictures can use (with attribution of
> > course)?
> >
> > I think I've watched every video presentation about Kafka I can find, but
> > if any of you who have presented want to share some common questions they
> > got or things they wish they'd included in their presentations please let
> > me know.
> >
> > Thanks,
> >
> > Chris
> >
>
Re: Atlanta AJUG presentation on Kafka March 19th
Posted by Jay Kreps <ja...@gmail.com>.
Also, this wiki has a pretty good collection of presentations which may
give you ideas. If you want the source ppt or omnigraffle for any of the
presentations we made let us know.
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+papers+and+presentations
-Jay
On Wed, Jan 16, 2013 at 7:26 AM, Chris Curtin <cu...@gmail.com>wrote:
> Hi,
>
> I'm going to be presenting an introduction to Kafka to the Atlanta Java
> User's Group on March 19th.
> http://www.meetup.com/atlantajug/events/99878712/
>
> This will be a 'hey, this is a cool technology to check out' presentation
> vs. a use case (since my day-job isn't using it in production yet). Anyone
> have a good architecture overview pictures can use (with attribution of
> course)?
>
> I think I've watched every video presentation about Kafka I can find, but
> if any of you who have presented want to share some common questions they
> got or things they wish they'd included in their presentations please let
> me know.
>
> Thanks,
>
> Chris
>
Re: Atlanta AJUG presentation on Kafka March 19th
Posted by Jun Rao <ju...@gmail.com>.
Chris,
That's great. Thanks for doing this.
For architecture slides, you may want to take a look at the Kafka ApacheCon
2011 slides in our wiki (just fixed the link).
Thanks,
Jun
On Wed, Jan 16, 2013 at 7:26 AM, Chris Curtin <cu...@gmail.com>wrote:
> Hi,
>
> I'm going to be presenting an introduction to Kafka to the Atlanta Java
> User's Group on March 19th.
> http://www.meetup.com/atlantajug/events/99878712/
>
> This will be a 'hey, this is a cool technology to check out' presentation
> vs. a use case (since my day-job isn't using it in production yet). Anyone
> have a good architecture overview pictures can use (with attribution of
> course)?
>
> I think I've watched every video presentation about Kafka I can find, but
> if any of you who have presented want to share some common questions they
> got or things they wish they'd included in their presentations please let
> me know.
>
> Thanks,
>
> Chris
>