You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Jun Rao <ju...@gmail.com> on 2012/05/29 18:01:11 UTC
wiki describing code path for major components
Hi,
To make it easy for new people to understand Kafka code, it would be useful
to document the code path for major components in Kafka. I started a
section in this wiki (
https://cwiki.apache.org/confluence/display/KAFKA/Index). I think we will
need to cover at least the following components. It would be great if
people who are familiar with those components can fill in the details.
- SocketServer
- RequestPurgatory
- LogManager and Log
- ReplicaManager
- ZookeeperConsumerConnector
Thanks,
Jun
Re: wiki describing code path for major components
Posted by Jay Kreps <ja...@gmail.com>.
Sounds great!
Sent from my iPhone
On Jun 9, 2012, at 7:08 AM, Guozhang Wang <gu...@linkedin.com> wrote:
> I would like to take the SocketServer (either old version or 0.8 version) if people are OK with it.
>
> Guozhang
> ________________________________________
> From: Jay Kreps [jay.kreps@gmail.com]
> Sent: Friday, June 08, 2012 2:06 PM
> To: kafka-dev@incubator.apache.org
> Subject: Re: wiki describing code path for major components
>
> I moved these onto a central page and added an architecture diagram:
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Internals
>
> On Tue, May 29, 2012 at 5:48 PM, Jay Kreps <ja...@gmail.com> wrote:
>
>> This is a great idea, I will write up the network layer (and request
>> purgatory stuff) and the log and log manager.
>>
>> -Jay
>>
>>
>> On Tue, May 29, 2012 at 9:23 AM, Neha Narkhede <ne...@gmail.com>wrote:
>>
>>> Jun,
>>>
>>> This is a great idea. I will take ZookeeperConsumerConnector and
>>> ReplicaManager.
>>>
>>> Thanks,
>>> Neha
>>>
>>> On Tue, May 29, 2012 at 9:01 AM, Jun Rao <ju...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> To make it easy for new people to understand Kafka code, it would be
>>> useful
>>>> to document the code path for major components in Kafka. I started a
>>>> section in this wiki (
>>>> https://cwiki.apache.org/confluence/display/KAFKA/Index). I think we
>>> will
>>>> need to cover at least the following components. It would be great if
>>>> people who are familiar with those components can fill in the details.
>>>>
>>>> - SocketServer
>>>> - RequestPurgatory
>>>> - LogManager and Log
>>>> - ReplicaManager
>>>> - ZookeeperConsumerConnector
>>>>
>>>> Thanks,
>>>>
>>>> Jun
>>>>
>>>
>>
>>
RE: wiki describing code path for major components
Posted by Guozhang Wang <gu...@linkedin.com>.
I would like to take the SocketServer (either old version or 0.8 version) if people are OK with it.
Guozhang
________________________________________
From: Jay Kreps [jay.kreps@gmail.com]
Sent: Friday, June 08, 2012 2:06 PM
To: kafka-dev@incubator.apache.org
Subject: Re: wiki describing code path for major components
I moved these onto a central page and added an architecture diagram:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Internals
On Tue, May 29, 2012 at 5:48 PM, Jay Kreps <ja...@gmail.com> wrote:
> This is a great idea, I will write up the network layer (and request
> purgatory stuff) and the log and log manager.
>
> -Jay
>
>
> On Tue, May 29, 2012 at 9:23 AM, Neha Narkhede <ne...@gmail.com>wrote:
>
>> Jun,
>>
>> This is a great idea. I will take ZookeeperConsumerConnector and
>> ReplicaManager.
>>
>> Thanks,
>> Neha
>>
>> On Tue, May 29, 2012 at 9:01 AM, Jun Rao <ju...@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > To make it easy for new people to understand Kafka code, it would be
>> useful
>> > to document the code path for major components in Kafka. I started a
>> > section in this wiki (
>> > https://cwiki.apache.org/confluence/display/KAFKA/Index). I think we
>> will
>> > need to cover at least the following components. It would be great if
>> > people who are familiar with those components can fill in the details.
>> >
>> > - SocketServer
>> > - RequestPurgatory
>> > - LogManager and Log
>> > - ReplicaManager
>> > - ZookeeperConsumerConnector
>> >
>> > Thanks,
>> >
>> > Jun
>> >
>>
>
>
Re: wiki describing code path for major components
Posted by Jay Kreps <ja...@gmail.com>.
I moved these onto a central page and added an architecture diagram:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Internals
On Tue, May 29, 2012 at 5:48 PM, Jay Kreps <ja...@gmail.com> wrote:
> This is a great idea, I will write up the network layer (and request
> purgatory stuff) and the log and log manager.
>
> -Jay
>
>
> On Tue, May 29, 2012 at 9:23 AM, Neha Narkhede <ne...@gmail.com>wrote:
>
>> Jun,
>>
>> This is a great idea. I will take ZookeeperConsumerConnector and
>> ReplicaManager.
>>
>> Thanks,
>> Neha
>>
>> On Tue, May 29, 2012 at 9:01 AM, Jun Rao <ju...@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > To make it easy for new people to understand Kafka code, it would be
>> useful
>> > to document the code path for major components in Kafka. I started a
>> > section in this wiki (
>> > https://cwiki.apache.org/confluence/display/KAFKA/Index). I think we
>> will
>> > need to cover at least the following components. It would be great if
>> > people who are familiar with those components can fill in the details.
>> >
>> > - SocketServer
>> > - RequestPurgatory
>> > - LogManager and Log
>> > - ReplicaManager
>> > - ZookeeperConsumerConnector
>> >
>> > Thanks,
>> >
>> > Jun
>> >
>>
>
>
Re: wiki describing code path for major components
Posted by Jay Kreps <ja...@gmail.com>.
This is a great idea, I will write up the network layer (and request
purgatory stuff) and the log and log manager.
-Jay
On Tue, May 29, 2012 at 9:23 AM, Neha Narkhede <ne...@gmail.com>wrote:
> Jun,
>
> This is a great idea. I will take ZookeeperConsumerConnector and
> ReplicaManager.
>
> Thanks,
> Neha
>
> On Tue, May 29, 2012 at 9:01 AM, Jun Rao <ju...@gmail.com> wrote:
>
> > Hi,
> >
> > To make it easy for new people to understand Kafka code, it would be
> useful
> > to document the code path for major components in Kafka. I started a
> > section in this wiki (
> > https://cwiki.apache.org/confluence/display/KAFKA/Index). I think we
> will
> > need to cover at least the following components. It would be great if
> > people who are familiar with those components can fill in the details.
> >
> > - SocketServer
> > - RequestPurgatory
> > - LogManager and Log
> > - ReplicaManager
> > - ZookeeperConsumerConnector
> >
> > Thanks,
> >
> > Jun
> >
>
Re: wiki describing code path for major components
Posted by Neha Narkhede <ne...@gmail.com>.
Jun,
This is a great idea. I will take ZookeeperConsumerConnector and
ReplicaManager.
Thanks,
Neha
On Tue, May 29, 2012 at 9:01 AM, Jun Rao <ju...@gmail.com> wrote:
> Hi,
>
> To make it easy for new people to understand Kafka code, it would be useful
> to document the code path for major components in Kafka. I started a
> section in this wiki (
> https://cwiki.apache.org/confluence/display/KAFKA/Index). I think we will
> need to cover at least the following components. It would be great if
> people who are familiar with those components can fill in the details.
>
> - SocketServer
> - RequestPurgatory
> - LogManager and Log
> - ReplicaManager
> - ZookeeperConsumerConnector
>
> Thanks,
>
> Jun
>