You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Jun Rao <ju...@gmail.com> on 2012/05/25 19:49:24 UTC
kafka replication work status
Hi,
Just want to give people a quick update on the progress on kafka
replication. We have more or less finalized the replication design (
https://cwiki.apache.org/confluence/display/KAFKA/kafka+Detailed+Replication+Design+V3)
and I encourage everyone to take a look and send us additional
feedback/comment.
Most of the remaining work for replication is listed as subtasks
under KAFKA-50. Currently, we have people working on the following jiras:
KAFKA-46: Commit thread, ReplicaFetcherThread for intra-cluster replication
KAFKA-336: Admin RPC between controller and broker
KAFKA-335: Embedded controller
KAFKA-339: Multifetch for follower
The first 3 jiras are blocking most of other jiras related to replication
and we hope to have those resolved in the next couple of weeks. Before
then, the following jiras can still be worked on more or
less independently and we welcome contribution from the community.
KAFKA-288: rework java producer API
KAFKA-289: reuse TopicData when sending produce request
KAFKA-329: Create topic support (revisit based on v3 design)
Thanks,
Jun
Re: kafka replication work status
Posted by Jun Rao <ju...@gmail.com>.
Prashanth,
As for timeline, the current plan is to close kafka-46 and a couple of
system tests (kafka-341 and kafka-306) first, hopefully within the next
week or two. This will allow the basic replication to work (not all failure
cases will be handled properly). In the remaining phase 2, we will continue
to improve the failure cases and move to the controller-based
implementation. We hope to finish most of the phase 2 around end of Jul.
Depending on how many of the advanced stuff we want to add in the first 0.8
release, we may need another month or so to complete phase 3.
Going forward, I will try to send out regular email updates on the progress
and remaining jiras related to replication.
As for KAFKA-329, it would be great if you can take on that.
Thanks,
Jun
On Mon, May 28, 2012 at 5:09 PM, Prashanth Menon <prashanth.menon1@gmail.com
> wrote:
> I can take a look at KAFKA-329 :) Just so everyone is aware, is there a
> timeline we'd like to hit as a community for getting the basic message
> replication stuff (phase 2?) for KAFKA-50? Knowing this will help everyone
> plan their time out a little better.
>
> Regards,
> Prashanth
>
> On Fri, May 25, 2012 at 1:49 PM, Jun Rao <ju...@gmail.com> wrote:
>
> > Hi,
> >
> > Just want to give people a quick update on the progress on kafka
> > replication. We have more or less finalized the replication design (
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/kafka+Detailed+Replication+Design+V3
> > )
> > and I encourage everyone to take a look and send us additional
> > feedback/comment.
> >
> > Most of the remaining work for replication is listed as subtasks
> > under KAFKA-50. Currently, we have people working on the following jiras:
> >
> > KAFKA-46: Commit thread, ReplicaFetcherThread for intra-cluster
> replication
> > KAFKA-336: Admin RPC between controller and broker
> > KAFKA-335: Embedded controller
> > KAFKA-339: Multifetch for follower
> >
> > The first 3 jiras are blocking most of other jiras related to replication
> > and we hope to have those resolved in the next couple of weeks. Before
> > then, the following jiras can still be worked on more or
> > less independently and we welcome contribution from the community.
> > KAFKA-288: rework java producer API
> > KAFKA-289: reuse TopicData when sending produce request
> > KAFKA-329: Create topic support (revisit based on v3 design)
> >
> > Thanks,
> >
> > Jun
> >
>
Re: kafka replication work status
Posted by Prashanth Menon <pr...@gmail.com>.
I can take a look at KAFKA-329 :) Just so everyone is aware, is there a
timeline we'd like to hit as a community for getting the basic message
replication stuff (phase 2?) for KAFKA-50? Knowing this will help everyone
plan their time out a little better.
Regards,
Prashanth
On Fri, May 25, 2012 at 1:49 PM, Jun Rao <ju...@gmail.com> wrote:
> Hi,
>
> Just want to give people a quick update on the progress on kafka
> replication. We have more or less finalized the replication design (
>
> https://cwiki.apache.org/confluence/display/KAFKA/kafka+Detailed+Replication+Design+V3
> )
> and I encourage everyone to take a look and send us additional
> feedback/comment.
>
> Most of the remaining work for replication is listed as subtasks
> under KAFKA-50. Currently, we have people working on the following jiras:
>
> KAFKA-46: Commit thread, ReplicaFetcherThread for intra-cluster replication
> KAFKA-336: Admin RPC between controller and broker
> KAFKA-335: Embedded controller
> KAFKA-339: Multifetch for follower
>
> The first 3 jiras are blocking most of other jiras related to replication
> and we hope to have those resolved in the next couple of weeks. Before
> then, the following jiras can still be worked on more or
> less independently and we welcome contribution from the community.
> KAFKA-288: rework java producer API
> KAFKA-289: reuse TopicData when sending produce request
> KAFKA-329: Create topic support (revisit based on v3 design)
>
> Thanks,
>
> Jun
>