You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Jun Rao <ju...@gmail.com> on 2013/10/08 02:33:37 UTC

Re: preparing for the 0.8 final release

Hi, Everyone,

I made another pass of the remaining jiras that we plan to fix in the 0.8
final release.

https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)

Do people agree with this list?

Joe,

I don't have good understanding of KAFKA-1018. Do you think this needs to
be fixed in 0.8 final?

Thanks,

Jun



On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:

> Hi, Everyone,
>
> We have been stabilizing the 0.8 branch since the beta1 release. I think
> we are getting close to an 0.8 final release. I made an initial list of the
> remaining jiras that should be fixed in 0.8.
>
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>
> 1. Do people agree with the list?
>
> 2. If the list is good, could people help contributing/reviewing the
> remaining jiras?
>
> Thanks,
>
> Jun
>

Re: preparing for the 0.8 final release

Posted by Jay Kreps <ja...@gmail.com>.
Sounds good to me too.

-Jay


On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:

> KAFKA-1008 has been checked into the 0.8 branch and needs to be manually
> double-committed to trunk. To avoid merging problems, I suggest that for
> all future changes in the 0.8 branch, we double commit them to trunk. Any
> objections?
>
> Thanks,
>
> Jun
>
>
> On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
>
> > Hi, Everyone,
> >
> > I made another pass of the remaining jiras that we plan to fix in the 0.8
> > final release.
> >
> >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >
> > Do people agree with this list?
> >
> > Joe,
> >
> > I don't have good understanding of KAFKA-1018. Do you think this needs to
> > be fixed in 0.8 final?
> >
> > Thanks,
> >
> > Jun
> >
> >
> >
> > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> >> Hi, Everyone,
> >>
> >> We have been stabilizing the 0.8 branch since the beta1 release. I think
> >> we are getting close to an 0.8 final release. I made an initial list of
> the
> >> remaining jiras that should be fixed in 0.8.
> >>
> >>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >>
> >> 1. Do people agree with the list?
> >>
> >> 2. If the list is good, could people help contributing/reviewing the
> >> remaining jiras?
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >
> >
>

Re: preparing for the 0.8 final release

Posted by Jay Kreps <ja...@gmail.com>.
Sounds good to me too.

-Jay


On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:

> KAFKA-1008 has been checked into the 0.8 branch and needs to be manually
> double-committed to trunk. To avoid merging problems, I suggest that for
> all future changes in the 0.8 branch, we double commit them to trunk. Any
> objections?
>
> Thanks,
>
> Jun
>
>
> On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
>
> > Hi, Everyone,
> >
> > I made another pass of the remaining jiras that we plan to fix in the 0.8
> > final release.
> >
> >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >
> > Do people agree with this list?
> >
> > Joe,
> >
> > I don't have good understanding of KAFKA-1018. Do you think this needs to
> > be fixed in 0.8 final?
> >
> > Thanks,
> >
> > Jun
> >
> >
> >
> > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> >> Hi, Everyone,
> >>
> >> We have been stabilizing the 0.8 branch since the beta1 release. I think
> >> we are getting close to an 0.8 final release. I made an initial list of
> the
> >> remaining jiras that should be fixed in 0.8.
> >>
> >>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >>
> >> 1. Do people agree with the list?
> >>
> >> 2. If the list is good, could people help contributing/reviewing the
> >> remaining jiras?
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >
> >
>

Re: preparing for the 0.8 final release

Posted by Neha Narkhede <ne...@gmail.com>.
Sounds good to me.

Thanks,
Neha


On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:

> KAFKA-1008 has been checked into the 0.8 branch and needs to be manually
> double-committed to trunk. To avoid merging problems, I suggest that for
> all future changes in the 0.8 branch, we double commit them to trunk. Any
> objections?
>
> Thanks,
>
> Jun
>
>
> On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
>
> > Hi, Everyone,
> >
> > I made another pass of the remaining jiras that we plan to fix in the 0.8
> > final release.
> >
> >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >
> > Do people agree with this list?
> >
> > Joe,
> >
> > I don't have good understanding of KAFKA-1018. Do you think this needs to
> > be fixed in 0.8 final?
> >
> > Thanks,
> >
> > Jun
> >
> >
> >
> > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> >> Hi, Everyone,
> >>
> >> We have been stabilizing the 0.8 branch since the beta1 release. I think
> >> we are getting close to an 0.8 final release. I made an initial list of
> the
> >> remaining jiras that should be fixed in 0.8.
> >>
> >>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >>
> >> 1. Do people agree with the list?
> >>
> >> 2. If the list is good, could people help contributing/reviewing the
> >> remaining jiras?
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >
> >
>

Re: Re: how can i use offset filter the msg

Posted by "kojie.fu" <ko...@gmail.com>.
Hi Guozhang.

thanks to your reply ,we manage this problem.

i will file it to the jira.

and now we compute the relation from the metris info.





kojie.fu

From: Guozhang Wang
Date: 2013-10-21 23:11
To: users@kafka.apache.org
Subject: Re: Re: how can i use offset filter the msg
Hi Kojie,

As Jun's FAQ indicates, today the only way you can do is to set/reset the
offset directly with Zookeeper. In addition, currently we do not have
correlations between offsets and timestamps, meaning that given a timestamp
you can not tell which message's offset is produced at around that time.
Can you file a jira about keeping track of the correlation between offsets
and timestamps?

Guozhang


On Mon, Oct 21, 2013 at 7:56 AM, Jun Rao <ju...@gmail.com> wrote:

> Just added an FAQ. Does that answer your question?
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-HowcanIrewindtheoffsetintheconsumer%3F
>
> Thanks,
>
> Jun
>
>
> On Sun, Oct 20, 2013 at 11:15 PM, kojie.fu <ko...@gmail.com> wrote:
>
> > hi guozhang,
> >     i mean ,in a  certain topic ，can i use the offset compute the msg's
> > time stamp(when the msg produced)？
> > in fact， i need an api look like:
> > setOffset(String topicId ,String groupId, long timeStamp){
> >
> > }
> > this method can be call by the comsumer ，and then  the offset will be
> > reset.
> > after call this method,the comsumer only comsume the msg occur after the
> > timeStamp.
> >
> > i am very sorry .I do not specify whether I describe clearly.
> > From: Guozhang Wang
> > Date: 2013-10-21 12:49
> > To: users@kafka.apache.org; kojie.fu
> > Subject: Re: how can i use offset filter the msg
> > Hello,
> >
> > I do not fully understanding your question, did you mean that you want to
> > know when a message with certain offset is produced?
> >
> >
> > Guozhang
> >
> >
> >
> >
> > On Sun, Oct 20, 2013 at 8:54 PM, kojie.fu <ko...@gmail.com> wrote:
> >
> > Hi all,
> > we have a scene like this:
> > A system produce  the msg ,B system comsumer the msg.
> > for some reason the B system unwork,and then  we need abandon the out of
> > date msg.
> > but now in our msg there is no time stamp.
> > how can i use the offset to do this.
> >
> > does kafka know when the msg produce?
> >
> > how can i  location the certain offset?
> >
> >
> >
> > thanks
> >
> >
> >
> > --
> >
> > -- Guozhang
> >
>



-- 
-- Guozhang

Re: Re: how can i use offset filter the msg

Posted by "kojie.fu" <ko...@gmail.com>.
yes，thanks.that is what  we wanted !

we now use the metrics info [LogEndOffset] compute the offset  some seconds ago in each partition.
and then use importZKOffsets ,It  works !

thanks very much!



kojie.fu

From: Jun Rao
Date: 2013-10-21 22:56
To: users@kafka.apache.org; kojie.fu
Subject: Re: Re: how can i use offset filter the msg
Just added an FAQ. Does that answer your question?


https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-HowcanIrewindtheoffsetintheconsumer%3F


Thanks,


Jun



On Sun, Oct 20, 2013 at 11:15 PM, kojie.fu <ko...@gmail.com> wrote:

hi guozhang,
    i mean ,in a  certain topic ，can i use the offset compute the msg's time stamp(when the msg produced)？
in fact， i need an api look like:
setOffset(String topicId ,String groupId, long timeStamp){

}
this method can be call by the comsumer ，and then  the offset will be reset.
after call this method,the comsumer only comsume the msg occur after the timeStamp.

i am very sorry .I do not specify whether I describe clearly.
From: Guozhang Wang
Date: 2013-10-21 12:49
To: users@kafka.apache.org; kojie.fu
Subject: Re: how can i use offset filter the msg

Hello,

I do not fully understanding your question, did you mean that you want to know when a message with certain offset is produced?


Guozhang




On Sun, Oct 20, 2013 at 8:54 PM, kojie.fu <ko...@gmail.com> wrote:

Hi all,
we have a scene like this:
A system produce  the msg ,B system comsumer the msg.
for some reason the B system unwork,and then  we need abandon the out of date msg.
but now in our msg there is no time stamp.
how can i use the offset to do this.

does kafka know when the msg produce?

how can i  location the certain offset?



thanks



--

-- Guozhang

Re: Re: how can i use offset filter the msg

Posted by Guozhang Wang <wa...@gmail.com>.
Hi Kojie,

As Jun's FAQ indicates, today the only way you can do is to set/reset the
offset directly with Zookeeper. In addition, currently we do not have
correlations between offsets and timestamps, meaning that given a timestamp
you can not tell which message's offset is produced at around that time.
Can you file a jira about keeping track of the correlation between offsets
and timestamps?

Guozhang


On Mon, Oct 21, 2013 at 7:56 AM, Jun Rao <ju...@gmail.com> wrote:

> Just added an FAQ. Does that answer your question?
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-HowcanIrewindtheoffsetintheconsumer%3F
>
> Thanks,
>
> Jun
>
>
> On Sun, Oct 20, 2013 at 11:15 PM, kojie.fu <ko...@gmail.com> wrote:
>
> > hi guozhang,
> >     i mean ,in a  certain topic ，can i use the offset compute the msg's
> > time stamp(when the msg produced)？
> > in fact， i need an api look like:
> > setOffset(String topicId ,String groupId, long timeStamp){
> >
> > }
> > this method can be call by the comsumer ，and then  the offset will be
> > reset.
> > after call this method,the comsumer only comsume the msg occur after the
> > timeStamp.
> >
> > i am very sorry .I do not specify whether I describe clearly.
> > From: Guozhang Wang
> > Date: 2013-10-21 12:49
> > To: users@kafka.apache.org; kojie.fu
> > Subject: Re: how can i use offset filter the msg
> > Hello,
> >
> > I do not fully understanding your question, did you mean that you want to
> > know when a message with certain offset is produced?
> >
> >
> > Guozhang
> >
> >
> >
> >
> > On Sun, Oct 20, 2013 at 8:54 PM, kojie.fu <ko...@gmail.com> wrote:
> >
> > Hi all,
> > we have a scene like this:
> > A system produce  the msg ,B system comsumer the msg.
> > for some reason the B system unwork,and then  we need abandon the out of
> > date msg.
> > but now in our msg there is no time stamp.
> > how can i use the offset to do this.
> >
> > does kafka know when the msg produce?
> >
> > how can i  location the certain offset?
> >
> >
> >
> > thanks
> >
> >
> >
> > --
> >
> > -- Guozhang
> >
>



-- 
-- Guozhang

Re: Re: how can i use offset filter the msg

Posted by Jun Rao <ju...@gmail.com>.
Just added an FAQ. Does that answer your question?

https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-HowcanIrewindtheoffsetintheconsumer%3F

Thanks,

Jun


On Sun, Oct 20, 2013 at 11:15 PM, kojie.fu <ko...@gmail.com> wrote:

> hi guozhang,
>     i mean ,in a  certain topic ，can i use the offset compute the msg's
> time stamp(when the msg produced)？
> in fact， i need an api look like:
> setOffset(String topicId ,String groupId, long timeStamp){
>
> }
> this method can be call by the comsumer ，and then  the offset will be
> reset.
> after call this method,the comsumer only comsume the msg occur after the
> timeStamp.
>
> i am very sorry .I do not specify whether I describe clearly.
> From: Guozhang Wang
> Date: 2013-10-21 12:49
> To: users@kafka.apache.org; kojie.fu
> Subject: Re: how can i use offset filter the msg
> Hello,
>
> I do not fully understanding your question, did you mean that you want to
> know when a message with certain offset is produced?
>
>
> Guozhang
>
>
>
>
> On Sun, Oct 20, 2013 at 8:54 PM, kojie.fu <ko...@gmail.com> wrote:
>
> Hi all,
> we have a scene like this:
> A system produce  the msg ,B system comsumer the msg.
> for some reason the B system unwork,and then  we need abandon the out of
> date msg.
> but now in our msg there is no time stamp.
> how can i use the offset to do this.
>
> does kafka know when the msg produce?
>
> how can i  location the certain offset?
>
>
>
> thanks
>
>
>
> --
>
> -- Guozhang
>

Re: Re: how can i use offset filter the msg

Posted by "kojie.fu" <ko...@gmail.com>.
hi guozhang,
    i mean ,in a  certain topic ，can i use the offset compute the msg's time stamp(when the msg produced)？
in fact， i need an api look like:
setOffset(String topicId ,String groupId, long timeStamp){

}
this method can be call by the comsumer ，and then  the offset will be reset.
after call this method,the comsumer only comsume the msg occur after the timeStamp.

i am very sorry .I do not specify whether I describe clearly.  
From: Guozhang Wang
Date: 2013-10-21 12:49
To: users@kafka.apache.org; kojie.fu
Subject: Re: how can i use offset filter the msg
Hello,

I do not fully understanding your question, did you mean that you want to know when a message with certain offset is produced?


Guozhang




On Sun, Oct 20, 2013 at 8:54 PM, kojie.fu <ko...@gmail.com> wrote:

Hi all,
we have a scene like this:
A system produce  the msg ,B system comsumer the msg.
for some reason the B system unwork,and then  we need abandon the out of date msg.
but now in our msg there is no time stamp.
how can i use the offset to do this.

does kafka know when the msg produce?

how can i  location the certain offset?



thanks



-- 

-- Guozhang

回复kojie.fu的问题

Posted by linghongbo008 <li...@gmail.com>.
 这个意识是这样的，我们有一个如下的场景：
一个A系统往kafka队列写消息，一个B系统负责从kafka消费消息
现可能由于某种原因B系统无法读取kafka的消息，此时A系统还一直在往队列写消息，所以当我们把B系统
修复起来后（比如说花了2天时间修复了），我们不想让B系统还从原来的offset读取数据，而是想从某个时间点开始读取数据，
不知道kafka有类似的 时间戳和offset对应的接口没？





From: Guozhang Wang
Date: 2013-10-21 12:49
To: users@kafka.apache.org; kojie.fu
Subject: Re: how can i use offset filter the msg
Hello,

I do not fully understanding your question, did you mean that you want to
know when a message with certain offset is produced?

Guozhang


On Sun, Oct 20, 2013 at 8:54 PM, kojie.fu <ko...@gmail.com> wrote:

> Hi all,
> we have a scene like this:
> A system produce  the msg ,B system comsumer the msg.
> for some reason the B system unwork,and then  we need abandon the out of
> date msg.
> but now in our msg there is no time stamp.
> how can i use the offset to do this.
>
> does kafka know when the msg produce?
>
> how can i  location the certain offset?
>
>
>
> thanks




-- 
-- Guozhang

Re: how can i use offset filter the msg

Posted by Guozhang Wang <wa...@gmail.com>.
Hello,

I do not fully understanding your question, did you mean that you want to
know when a message with certain offset is produced?

Guozhang


On Sun, Oct 20, 2013 at 8:54 PM, kojie.fu <ko...@gmail.com> wrote:

> Hi all,
> we have a scene like this:
> A system produce  the msg ,B system comsumer the msg.
> for some reason the B system unwork,and then  we need abandon the out of
> date msg.
> but now in our msg there is no time stamp.
> how can i use the offset to do this.
>
> does kafka know when the msg produce?
>
> how can i  location the certain offset?
>
>
>
> thanks




-- 
-- Guozhang

how can i use offset filter the msg

Posted by "kojie.fu" <ko...@gmail.com>.
Hi all,
we have a scene like this:
A system produce  the msg ,B system comsumer the msg.
for some reason the B system unwork,and then  we need abandon the out of date msg.
but now in our msg there is no time stamp.
how can i use the offset to do this.

does kafka know when the msg produce?

how can i  location the certain offset?



thanks

how can i use offset filter the msg

Posted by "kojie.fu" <ko...@gmail.com>.
Hi all,
we have a scene like this:
A system produce  the msg ,B system comsumer the msg.
for some reason the B system unwork,and then  we need abandon the out of date msg.
but now in our msg there is no time stamp.
how can i use the offset to do this.

does kafka know when the msg produce?

how can i  location the certain offset?



thanks

Re: preparing for the 0.8 final release

Posted by Jun Rao <ju...@gmail.com>.
The fetch thread uses multi-fetch. Have you looked at
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whymessagesaredelayedinmyconsumer%3F,
which may be related to your issue.

Thanks,

Jun


On Sun, Oct 20, 2013 at 2:48 AM, Jason Rosenberg <jb...@squareup.com> wrote:

> Ok,
>
> So here's an outline of what I think seems to have happened.
>
> I have a consumer, that uses a filter to consume a large number of topics
> (e.g. several hundred).  Each topic has only a single partition.
>
> It normally has no trouble keeping up processing all messages on all
> topics.   However, we had a case a couple days ago where it seemed to hang,
> and not consume anything for several hours.  I restarted the consumer (and
> now I've updated it from 0.8-beta1 to 0.8-latest-HEAD).  Data is flowing
> again, but some topics are seeming to take much longer than others to catch
> up.  The slow ones seem to be the topics that have more data than others (a
> loose theory at present).
>
> Does that make sense?  If I understand things correctly, the consumer will
> fetch chunks of data from each topic/partition, in order, in a big loop?
>  So if it has caught up with most of the topics, will it waste time
> re-polling all those (and getting nothing) before coming back to the topics
> that are lagging?  Perhaps having a larger fetch size would help here?
>
> Jason
>
>
> On Sat, Oct 19, 2013 at 6:24 PM, Jason Rosenberg <jb...@squareup.com> wrote:
>
> > I'll try to, next time it hangs!
> >
> >
> > On Sat, Oct 19, 2013 at 4:04 PM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
> >
> >> Can you send around a thread dump of the halted consumer process?
> >>
> >>
> >>
> >> On Sat, Oct 19, 2013 at 12:16 PM, Jason Rosenberg <jb...@squareup.com>
> >> wrote:
> >>
> >> > The latest HEAD does seem to solve one issue, where a new topic being
> >> > created after the consumer is started, would not be consumed.
> >> >
> >> > But the bigger issue is that we have a couple different consumers both
> >> > consuming the same set of topics (under different groupids), and
> hanging
> >> > after a while (both hanging at about the same point).  The topics in
> >> each
> >> > case are selected with a filter (actually a relatively large number of
> >> > topics, some of which are newly created over time).  I'm still not
> sure
> >> > whether the new version is solving this issue (since it was a rare
> >> > transient thing anyway).
> >> >
> >> > Thanks,
> >> >
> >> > Jason
> >> >
> >> >
> >> > On Sat, Oct 19, 2013 at 2:03 AM, Jun Rao <ju...@gmail.com> wrote:
> >> >
> >> > > Yes, 0.8 will be released from the HEAD of the 0.8 branch. Is the
> >> problem
> >> > > with consuming new topics or topics whose partitions are increased?
> If
> >> > so,
> >> > > see KAFKA-1030 and KAFKA-1075.
> >> > >
> >> > > Thanks,
> >> > >
> >> > > Jun
> >> > >
> >> > >
> >> > > On Fri, Oct 18, 2013 at 4:03 PM, Jason Rosenberg <jb...@squareup.com>
> >> > wrote:
> >> > >
> >> > > > Will the 0.8 release come from the HEAD of the 0.8 branch?  I'd
> >> like to
> >> > > > experiment with it, to see if it solves some of the issues I'm
> >> seeing,
> >> > > with
> >> > > > consumers refusing to consume new messages.  We've been using the
> >> beta1
> >> > > > version.
> >> > > >
> >> > > > I remember mention there was a Jira issues along these lines,
> which
> >> was
> >> > > > fixed post 0.8-beta1.  Which issue was that (I'd like to see if it
> >> > > matches
> >> > > > what I'm seeing).
> >> > > >
> >> > > > Thanks,
> >> > > >
> >> > > > Jason
> >> > > >
> >> > > >
> >> > > > On Wed, Oct 9, 2013 at 8:04 PM, Jay Kreps <ja...@gmail.com>
> >> wrote:
> >> > > >
> >> > > > > I uploaded a patch against trunk which also fixes KAFKA-1036,
> the
> >> > other
> >> > > > > knows windows issue. Review appreciated. Should be an easy one.
> >> > > > >
> >> > > > > https://issues.apache.org/jira/browse/KAFKA-1008
> >> > > > >
> >> > > > > -Jay
> >> > > > >
> >> > > > >
> >> > > > > On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com>
> wrote:
> >> > > > >
> >> > > > > > KAFKA-1008 has been checked into the 0.8 branch and needs to
> be
> >> > > > manually
> >> > > > > > double-committed to trunk. To avoid merging problems, I
> suggest
> >> > that
> >> > > > for
> >> > > > > > all future changes in the 0.8 branch, we double commit them to
> >> > trunk.
> >> > > > Any
> >> > > > > > objections?
> >> > > > > >
> >> > > > > > Thanks,
> >> > > > > >
> >> > > > > > Jun
> >> > > > > >
> >> > > > > >
> >> > > > > > On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com>
> >> wrote:
> >> > > > > >
> >> > > > > > > Hi, Everyone,
> >> > > > > > >
> >> > > > > > > I made another pass of the remaining jiras that we plan to
> >> fix in
> >> > > the
> >> > > > > 0.8
> >> > > > > > > final release.
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >> > > > > > >
> >> > > > > > > Do people agree with this list?
> >> > > > > > >
> >> > > > > > > Joe,
> >> > > > > > >
> >> > > > > > > I don't have good understanding of KAFKA-1018. Do you think
> >> this
> >> > > > needs
> >> > > > > to
> >> > > > > > > be fixed in 0.8 final?
> >> > > > > > >
> >> > > > > > > Thanks,
> >> > > > > > >
> >> > > > > > > Jun
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com>
> >> > wrote:
> >> > > > > > >
> >> > > > > > >> Hi, Everyone,
> >> > > > > > >>
> >> > > > > > >> We have been stabilizing the 0.8 branch since the beta1
> >> > release. I
> >> > > > > think
> >> > > > > > >> we are getting close to an 0.8 final release. I made an
> >> initial
> >> > > list
> >> > > > > of
> >> > > > > > the
> >> > > > > > >> remaining jiras that should be fixed in 0.8.
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >> > > > > > >>
> >> > > > > > >> 1. Do people agree with the list?
> >> > > > > > >>
> >> > > > > > >> 2. If the list is good, could people help
> >> contributing/reviewing
> >> > > the
> >> > > > > > >> remaining jiras?
> >> > > > > > >>
> >> > > > > > >> Thanks,
> >> > > > > > >>
> >> > > > > > >> Jun
> >> > > > > > >>
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Re: preparing for the 0.8 final release

Posted by Jun Rao <ju...@gmail.com>.
The fetch thread uses multi-fetch. Have you looked at
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whymessagesaredelayedinmyconsumer%3F,
which may be related to your issue.

Thanks,

Jun


On Sun, Oct 20, 2013 at 2:48 AM, Jason Rosenberg <jb...@squareup.com> wrote:

> Ok,
>
> So here's an outline of what I think seems to have happened.
>
> I have a consumer, that uses a filter to consume a large number of topics
> (e.g. several hundred).  Each topic has only a single partition.
>
> It normally has no trouble keeping up processing all messages on all
> topics.   However, we had a case a couple days ago where it seemed to hang,
> and not consume anything for several hours.  I restarted the consumer (and
> now I've updated it from 0.8-beta1 to 0.8-latest-HEAD).  Data is flowing
> again, but some topics are seeming to take much longer than others to catch
> up.  The slow ones seem to be the topics that have more data than others (a
> loose theory at present).
>
> Does that make sense?  If I understand things correctly, the consumer will
> fetch chunks of data from each topic/partition, in order, in a big loop?
>  So if it has caught up with most of the topics, will it waste time
> re-polling all those (and getting nothing) before coming back to the topics
> that are lagging?  Perhaps having a larger fetch size would help here?
>
> Jason
>
>
> On Sat, Oct 19, 2013 at 6:24 PM, Jason Rosenberg <jb...@squareup.com> wrote:
>
> > I'll try to, next time it hangs!
> >
> >
> > On Sat, Oct 19, 2013 at 4:04 PM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
> >
> >> Can you send around a thread dump of the halted consumer process?
> >>
> >>
> >>
> >> On Sat, Oct 19, 2013 at 12:16 PM, Jason Rosenberg <jb...@squareup.com>
> >> wrote:
> >>
> >> > The latest HEAD does seem to solve one issue, where a new topic being
> >> > created after the consumer is started, would not be consumed.
> >> >
> >> > But the bigger issue is that we have a couple different consumers both
> >> > consuming the same set of topics (under different groupids), and
> hanging
> >> > after a while (both hanging at about the same point).  The topics in
> >> each
> >> > case are selected with a filter (actually a relatively large number of
> >> > topics, some of which are newly created over time).  I'm still not
> sure
> >> > whether the new version is solving this issue (since it was a rare
> >> > transient thing anyway).
> >> >
> >> > Thanks,
> >> >
> >> > Jason
> >> >
> >> >
> >> > On Sat, Oct 19, 2013 at 2:03 AM, Jun Rao <ju...@gmail.com> wrote:
> >> >
> >> > > Yes, 0.8 will be released from the HEAD of the 0.8 branch. Is the
> >> problem
> >> > > with consuming new topics or topics whose partitions are increased?
> If
> >> > so,
> >> > > see KAFKA-1030 and KAFKA-1075.
> >> > >
> >> > > Thanks,
> >> > >
> >> > > Jun
> >> > >
> >> > >
> >> > > On Fri, Oct 18, 2013 at 4:03 PM, Jason Rosenberg <jb...@squareup.com>
> >> > wrote:
> >> > >
> >> > > > Will the 0.8 release come from the HEAD of the 0.8 branch?  I'd
> >> like to
> >> > > > experiment with it, to see if it solves some of the issues I'm
> >> seeing,
> >> > > with
> >> > > > consumers refusing to consume new messages.  We've been using the
> >> beta1
> >> > > > version.
> >> > > >
> >> > > > I remember mention there was a Jira issues along these lines,
> which
> >> was
> >> > > > fixed post 0.8-beta1.  Which issue was that (I'd like to see if it
> >> > > matches
> >> > > > what I'm seeing).
> >> > > >
> >> > > > Thanks,
> >> > > >
> >> > > > Jason
> >> > > >
> >> > > >
> >> > > > On Wed, Oct 9, 2013 at 8:04 PM, Jay Kreps <ja...@gmail.com>
> >> wrote:
> >> > > >
> >> > > > > I uploaded a patch against trunk which also fixes KAFKA-1036,
> the
> >> > other
> >> > > > > knows windows issue. Review appreciated. Should be an easy one.
> >> > > > >
> >> > > > > https://issues.apache.org/jira/browse/KAFKA-1008
> >> > > > >
> >> > > > > -Jay
> >> > > > >
> >> > > > >
> >> > > > > On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com>
> wrote:
> >> > > > >
> >> > > > > > KAFKA-1008 has been checked into the 0.8 branch and needs to
> be
> >> > > > manually
> >> > > > > > double-committed to trunk. To avoid merging problems, I
> suggest
> >> > that
> >> > > > for
> >> > > > > > all future changes in the 0.8 branch, we double commit them to
> >> > trunk.
> >> > > > Any
> >> > > > > > objections?
> >> > > > > >
> >> > > > > > Thanks,
> >> > > > > >
> >> > > > > > Jun
> >> > > > > >
> >> > > > > >
> >> > > > > > On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com>
> >> wrote:
> >> > > > > >
> >> > > > > > > Hi, Everyone,
> >> > > > > > >
> >> > > > > > > I made another pass of the remaining jiras that we plan to
> >> fix in
> >> > > the
> >> > > > > 0.8
> >> > > > > > > final release.
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >> > > > > > >
> >> > > > > > > Do people agree with this list?
> >> > > > > > >
> >> > > > > > > Joe,
> >> > > > > > >
> >> > > > > > > I don't have good understanding of KAFKA-1018. Do you think
> >> this
> >> > > > needs
> >> > > > > to
> >> > > > > > > be fixed in 0.8 final?
> >> > > > > > >
> >> > > > > > > Thanks,
> >> > > > > > >
> >> > > > > > > Jun
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com>
> >> > wrote:
> >> > > > > > >
> >> > > > > > >> Hi, Everyone,
> >> > > > > > >>
> >> > > > > > >> We have been stabilizing the 0.8 branch since the beta1
> >> > release. I
> >> > > > > think
> >> > > > > > >> we are getting close to an 0.8 final release. I made an
> >> initial
> >> > > list
> >> > > > > of
> >> > > > > > the
> >> > > > > > >> remaining jiras that should be fixed in 0.8.
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >> > > > > > >>
> >> > > > > > >> 1. Do people agree with the list?
> >> > > > > > >>
> >> > > > > > >> 2. If the list is good, could people help
> >> contributing/reviewing
> >> > > the
> >> > > > > > >> remaining jiras?
> >> > > > > > >>
> >> > > > > > >> Thanks,
> >> > > > > > >>
> >> > > > > > >> Jun
> >> > > > > > >>
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Re: preparing for the 0.8 final release

Posted by Jason Rosenberg <jb...@squareup.com>.
Ok,

So here's an outline of what I think seems to have happened.

I have a consumer, that uses a filter to consume a large number of topics
(e.g. several hundred).  Each topic has only a single partition.

It normally has no trouble keeping up processing all messages on all
topics.   However, we had a case a couple days ago where it seemed to hang,
and not consume anything for several hours.  I restarted the consumer (and
now I've updated it from 0.8-beta1 to 0.8-latest-HEAD).  Data is flowing
again, but some topics are seeming to take much longer than others to catch
up.  The slow ones seem to be the topics that have more data than others (a
loose theory at present).

Does that make sense?  If I understand things correctly, the consumer will
fetch chunks of data from each topic/partition, in order, in a big loop?
 So if it has caught up with most of the topics, will it waste time
re-polling all those (and getting nothing) before coming back to the topics
that are lagging?  Perhaps having a larger fetch size would help here?

Jason


On Sat, Oct 19, 2013 at 6:24 PM, Jason Rosenberg <jb...@squareup.com> wrote:

> I'll try to, next time it hangs!
>
>
> On Sat, Oct 19, 2013 at 4:04 PM, Neha Narkhede <ne...@gmail.com>wrote:
>
>> Can you send around a thread dump of the halted consumer process?
>>
>>
>>
>> On Sat, Oct 19, 2013 at 12:16 PM, Jason Rosenberg <jb...@squareup.com>
>> wrote:
>>
>> > The latest HEAD does seem to solve one issue, where a new topic being
>> > created after the consumer is started, would not be consumed.
>> >
>> > But the bigger issue is that we have a couple different consumers both
>> > consuming the same set of topics (under different groupids), and hanging
>> > after a while (both hanging at about the same point).  The topics in
>> each
>> > case are selected with a filter (actually a relatively large number of
>> > topics, some of which are newly created over time).  I'm still not sure
>> > whether the new version is solving this issue (since it was a rare
>> > transient thing anyway).
>> >
>> > Thanks,
>> >
>> > Jason
>> >
>> >
>> > On Sat, Oct 19, 2013 at 2:03 AM, Jun Rao <ju...@gmail.com> wrote:
>> >
>> > > Yes, 0.8 will be released from the HEAD of the 0.8 branch. Is the
>> problem
>> > > with consuming new topics or topics whose partitions are increased? If
>> > so,
>> > > see KAFKA-1030 and KAFKA-1075.
>> > >
>> > > Thanks,
>> > >
>> > > Jun
>> > >
>> > >
>> > > On Fri, Oct 18, 2013 at 4:03 PM, Jason Rosenberg <jb...@squareup.com>
>> > wrote:
>> > >
>> > > > Will the 0.8 release come from the HEAD of the 0.8 branch?  I'd
>> like to
>> > > > experiment with it, to see if it solves some of the issues I'm
>> seeing,
>> > > with
>> > > > consumers refusing to consume new messages.  We've been using the
>> beta1
>> > > > version.
>> > > >
>> > > > I remember mention there was a Jira issues along these lines, which
>> was
>> > > > fixed post 0.8-beta1.  Which issue was that (I'd like to see if it
>> > > matches
>> > > > what I'm seeing).
>> > > >
>> > > > Thanks,
>> > > >
>> > > > Jason
>> > > >
>> > > >
>> > > > On Wed, Oct 9, 2013 at 8:04 PM, Jay Kreps <ja...@gmail.com>
>> wrote:
>> > > >
>> > > > > I uploaded a patch against trunk which also fixes KAFKA-1036, the
>> > other
>> > > > > knows windows issue. Review appreciated. Should be an easy one.
>> > > > >
>> > > > > https://issues.apache.org/jira/browse/KAFKA-1008
>> > > > >
>> > > > > -Jay
>> > > > >
>> > > > >
>> > > > > On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:
>> > > > >
>> > > > > > KAFKA-1008 has been checked into the 0.8 branch and needs to be
>> > > > manually
>> > > > > > double-committed to trunk. To avoid merging problems, I suggest
>> > that
>> > > > for
>> > > > > > all future changes in the 0.8 branch, we double commit them to
>> > trunk.
>> > > > Any
>> > > > > > objections?
>> > > > > >
>> > > > > > Thanks,
>> > > > > >
>> > > > > > Jun
>> > > > > >
>> > > > > >
>> > > > > > On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com>
>> wrote:
>> > > > > >
>> > > > > > > Hi, Everyone,
>> > > > > > >
>> > > > > > > I made another pass of the remaining jiras that we plan to
>> fix in
>> > > the
>> > > > > 0.8
>> > > > > > > final release.
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>> > > > > > >
>> > > > > > > Do people agree with this list?
>> > > > > > >
>> > > > > > > Joe,
>> > > > > > >
>> > > > > > > I don't have good understanding of KAFKA-1018. Do you think
>> this
>> > > > needs
>> > > > > to
>> > > > > > > be fixed in 0.8 final?
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > >
>> > > > > > > Jun
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com>
>> > wrote:
>> > > > > > >
>> > > > > > >> Hi, Everyone,
>> > > > > > >>
>> > > > > > >> We have been stabilizing the 0.8 branch since the beta1
>> > release. I
>> > > > > think
>> > > > > > >> we are getting close to an 0.8 final release. I made an
>> initial
>> > > list
>> > > > > of
>> > > > > > the
>> > > > > > >> remaining jiras that should be fixed in 0.8.
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>> > > > > > >>
>> > > > > > >> 1. Do people agree with the list?
>> > > > > > >>
>> > > > > > >> 2. If the list is good, could people help
>> contributing/reviewing
>> > > the
>> > > > > > >> remaining jiras?
>> > > > > > >>
>> > > > > > >> Thanks,
>> > > > > > >>
>> > > > > > >> Jun
>> > > > > > >>
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Re: preparing for the 0.8 final release

Posted by Jason Rosenberg <jb...@squareup.com>.
Ok,

So here's an outline of what I think seems to have happened.

I have a consumer, that uses a filter to consume a large number of topics
(e.g. several hundred).  Each topic has only a single partition.

It normally has no trouble keeping up processing all messages on all
topics.   However, we had a case a couple days ago where it seemed to hang,
and not consume anything for several hours.  I restarted the consumer (and
now I've updated it from 0.8-beta1 to 0.8-latest-HEAD).  Data is flowing
again, but some topics are seeming to take much longer than others to catch
up.  The slow ones seem to be the topics that have more data than others (a
loose theory at present).

Does that make sense?  If I understand things correctly, the consumer will
fetch chunks of data from each topic/partition, in order, in a big loop?
 So if it has caught up with most of the topics, will it waste time
re-polling all those (and getting nothing) before coming back to the topics
that are lagging?  Perhaps having a larger fetch size would help here?

Jason


On Sat, Oct 19, 2013 at 6:24 PM, Jason Rosenberg <jb...@squareup.com> wrote:

> I'll try to, next time it hangs!
>
>
> On Sat, Oct 19, 2013 at 4:04 PM, Neha Narkhede <ne...@gmail.com>wrote:
>
>> Can you send around a thread dump of the halted consumer process?
>>
>>
>>
>> On Sat, Oct 19, 2013 at 12:16 PM, Jason Rosenberg <jb...@squareup.com>
>> wrote:
>>
>> > The latest HEAD does seem to solve one issue, where a new topic being
>> > created after the consumer is started, would not be consumed.
>> >
>> > But the bigger issue is that we have a couple different consumers both
>> > consuming the same set of topics (under different groupids), and hanging
>> > after a while (both hanging at about the same point).  The topics in
>> each
>> > case are selected with a filter (actually a relatively large number of
>> > topics, some of which are newly created over time).  I'm still not sure
>> > whether the new version is solving this issue (since it was a rare
>> > transient thing anyway).
>> >
>> > Thanks,
>> >
>> > Jason
>> >
>> >
>> > On Sat, Oct 19, 2013 at 2:03 AM, Jun Rao <ju...@gmail.com> wrote:
>> >
>> > > Yes, 0.8 will be released from the HEAD of the 0.8 branch. Is the
>> problem
>> > > with consuming new topics or topics whose partitions are increased? If
>> > so,
>> > > see KAFKA-1030 and KAFKA-1075.
>> > >
>> > > Thanks,
>> > >
>> > > Jun
>> > >
>> > >
>> > > On Fri, Oct 18, 2013 at 4:03 PM, Jason Rosenberg <jb...@squareup.com>
>> > wrote:
>> > >
>> > > > Will the 0.8 release come from the HEAD of the 0.8 branch?  I'd
>> like to
>> > > > experiment with it, to see if it solves some of the issues I'm
>> seeing,
>> > > with
>> > > > consumers refusing to consume new messages.  We've been using the
>> beta1
>> > > > version.
>> > > >
>> > > > I remember mention there was a Jira issues along these lines, which
>> was
>> > > > fixed post 0.8-beta1.  Which issue was that (I'd like to see if it
>> > > matches
>> > > > what I'm seeing).
>> > > >
>> > > > Thanks,
>> > > >
>> > > > Jason
>> > > >
>> > > >
>> > > > On Wed, Oct 9, 2013 at 8:04 PM, Jay Kreps <ja...@gmail.com>
>> wrote:
>> > > >
>> > > > > I uploaded a patch against trunk which also fixes KAFKA-1036, the
>> > other
>> > > > > knows windows issue. Review appreciated. Should be an easy one.
>> > > > >
>> > > > > https://issues.apache.org/jira/browse/KAFKA-1008
>> > > > >
>> > > > > -Jay
>> > > > >
>> > > > >
>> > > > > On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:
>> > > > >
>> > > > > > KAFKA-1008 has been checked into the 0.8 branch and needs to be
>> > > > manually
>> > > > > > double-committed to trunk. To avoid merging problems, I suggest
>> > that
>> > > > for
>> > > > > > all future changes in the 0.8 branch, we double commit them to
>> > trunk.
>> > > > Any
>> > > > > > objections?
>> > > > > >
>> > > > > > Thanks,
>> > > > > >
>> > > > > > Jun
>> > > > > >
>> > > > > >
>> > > > > > On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com>
>> wrote:
>> > > > > >
>> > > > > > > Hi, Everyone,
>> > > > > > >
>> > > > > > > I made another pass of the remaining jiras that we plan to
>> fix in
>> > > the
>> > > > > 0.8
>> > > > > > > final release.
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>> > > > > > >
>> > > > > > > Do people agree with this list?
>> > > > > > >
>> > > > > > > Joe,
>> > > > > > >
>> > > > > > > I don't have good understanding of KAFKA-1018. Do you think
>> this
>> > > > needs
>> > > > > to
>> > > > > > > be fixed in 0.8 final?
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > >
>> > > > > > > Jun
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com>
>> > wrote:
>> > > > > > >
>> > > > > > >> Hi, Everyone,
>> > > > > > >>
>> > > > > > >> We have been stabilizing the 0.8 branch since the beta1
>> > release. I
>> > > > > think
>> > > > > > >> we are getting close to an 0.8 final release. I made an
>> initial
>> > > list
>> > > > > of
>> > > > > > the
>> > > > > > >> remaining jiras that should be fixed in 0.8.
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>> > > > > > >>
>> > > > > > >> 1. Do people agree with the list?
>> > > > > > >>
>> > > > > > >> 2. If the list is good, could people help
>> contributing/reviewing
>> > > the
>> > > > > > >> remaining jiras?
>> > > > > > >>
>> > > > > > >> Thanks,
>> > > > > > >>
>> > > > > > >> Jun
>> > > > > > >>
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Re: preparing for the 0.8 final release

Posted by Jason Rosenberg <jb...@squareup.com>.
I'll try to, next time it hangs!


On Sat, Oct 19, 2013 at 4:04 PM, Neha Narkhede <ne...@gmail.com>wrote:

> Can you send around a thread dump of the halted consumer process?
>
>
>
> On Sat, Oct 19, 2013 at 12:16 PM, Jason Rosenberg <jb...@squareup.com>
> wrote:
>
> > The latest HEAD does seem to solve one issue, where a new topic being
> > created after the consumer is started, would not be consumed.
> >
> > But the bigger issue is that we have a couple different consumers both
> > consuming the same set of topics (under different groupids), and hanging
> > after a while (both hanging at about the same point).  The topics in each
> > case are selected with a filter (actually a relatively large number of
> > topics, some of which are newly created over time).  I'm still not sure
> > whether the new version is solving this issue (since it was a rare
> > transient thing anyway).
> >
> > Thanks,
> >
> > Jason
> >
> >
> > On Sat, Oct 19, 2013 at 2:03 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> > > Yes, 0.8 will be released from the HEAD of the 0.8 branch. Is the
> problem
> > > with consuming new topics or topics whose partitions are increased? If
> > so,
> > > see KAFKA-1030 and KAFKA-1075.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Fri, Oct 18, 2013 at 4:03 PM, Jason Rosenberg <jb...@squareup.com>
> > wrote:
> > >
> > > > Will the 0.8 release come from the HEAD of the 0.8 branch?  I'd like
> to
> > > > experiment with it, to see if it solves some of the issues I'm
> seeing,
> > > with
> > > > consumers refusing to consume new messages.  We've been using the
> beta1
> > > > version.
> > > >
> > > > I remember mention there was a Jira issues along these lines, which
> was
> > > > fixed post 0.8-beta1.  Which issue was that (I'd like to see if it
> > > matches
> > > > what I'm seeing).
> > > >
> > > > Thanks,
> > > >
> > > > Jason
> > > >
> > > >
> > > > On Wed, Oct 9, 2013 at 8:04 PM, Jay Kreps <ja...@gmail.com>
> wrote:
> > > >
> > > > > I uploaded a patch against trunk which also fixes KAFKA-1036, the
> > other
> > > > > knows windows issue. Review appreciated. Should be an easy one.
> > > > >
> > > > > https://issues.apache.org/jira/browse/KAFKA-1008
> > > > >
> > > > > -Jay
> > > > >
> > > > >
> > > > > On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:
> > > > >
> > > > > > KAFKA-1008 has been checked into the 0.8 branch and needs to be
> > > > manually
> > > > > > double-committed to trunk. To avoid merging problems, I suggest
> > that
> > > > for
> > > > > > all future changes in the 0.8 branch, we double commit them to
> > trunk.
> > > > Any
> > > > > > objections?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > >
> > > > > > On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com>
> wrote:
> > > > > >
> > > > > > > Hi, Everyone,
> > > > > > >
> > > > > > > I made another pass of the remaining jiras that we plan to fix
> in
> > > the
> > > > > 0.8
> > > > > > > final release.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > > > > >
> > > > > > > Do people agree with this list?
> > > > > > >
> > > > > > > Joe,
> > > > > > >
> > > > > > > I don't have good understanding of KAFKA-1018. Do you think
> this
> > > > needs
> > > > > to
> > > > > > > be fixed in 0.8 final?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com>
> > wrote:
> > > > > > >
> > > > > > >> Hi, Everyone,
> > > > > > >>
> > > > > > >> We have been stabilizing the 0.8 branch since the beta1
> > release. I
> > > > > think
> > > > > > >> we are getting close to an 0.8 final release. I made an
> initial
> > > list
> > > > > of
> > > > > > the
> > > > > > >> remaining jiras that should be fixed in 0.8.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > > > > >>
> > > > > > >> 1. Do people agree with the list?
> > > > > > >>
> > > > > > >> 2. If the list is good, could people help
> contributing/reviewing
> > > the
> > > > > > >> remaining jiras?
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >>
> > > > > > >> Jun
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: preparing for the 0.8 final release

Posted by Jason Rosenberg <jb...@squareup.com>.
I'll try to, next time it hangs!


On Sat, Oct 19, 2013 at 4:04 PM, Neha Narkhede <ne...@gmail.com>wrote:

> Can you send around a thread dump of the halted consumer process?
>
>
>
> On Sat, Oct 19, 2013 at 12:16 PM, Jason Rosenberg <jb...@squareup.com>
> wrote:
>
> > The latest HEAD does seem to solve one issue, where a new topic being
> > created after the consumer is started, would not be consumed.
> >
> > But the bigger issue is that we have a couple different consumers both
> > consuming the same set of topics (under different groupids), and hanging
> > after a while (both hanging at about the same point).  The topics in each
> > case are selected with a filter (actually a relatively large number of
> > topics, some of which are newly created over time).  I'm still not sure
> > whether the new version is solving this issue (since it was a rare
> > transient thing anyway).
> >
> > Thanks,
> >
> > Jason
> >
> >
> > On Sat, Oct 19, 2013 at 2:03 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> > > Yes, 0.8 will be released from the HEAD of the 0.8 branch. Is the
> problem
> > > with consuming new topics or topics whose partitions are increased? If
> > so,
> > > see KAFKA-1030 and KAFKA-1075.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Fri, Oct 18, 2013 at 4:03 PM, Jason Rosenberg <jb...@squareup.com>
> > wrote:
> > >
> > > > Will the 0.8 release come from the HEAD of the 0.8 branch?  I'd like
> to
> > > > experiment with it, to see if it solves some of the issues I'm
> seeing,
> > > with
> > > > consumers refusing to consume new messages.  We've been using the
> beta1
> > > > version.
> > > >
> > > > I remember mention there was a Jira issues along these lines, which
> was
> > > > fixed post 0.8-beta1.  Which issue was that (I'd like to see if it
> > > matches
> > > > what I'm seeing).
> > > >
> > > > Thanks,
> > > >
> > > > Jason
> > > >
> > > >
> > > > On Wed, Oct 9, 2013 at 8:04 PM, Jay Kreps <ja...@gmail.com>
> wrote:
> > > >
> > > > > I uploaded a patch against trunk which also fixes KAFKA-1036, the
> > other
> > > > > knows windows issue. Review appreciated. Should be an easy one.
> > > > >
> > > > > https://issues.apache.org/jira/browse/KAFKA-1008
> > > > >
> > > > > -Jay
> > > > >
> > > > >
> > > > > On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:
> > > > >
> > > > > > KAFKA-1008 has been checked into the 0.8 branch and needs to be
> > > > manually
> > > > > > double-committed to trunk. To avoid merging problems, I suggest
> > that
> > > > for
> > > > > > all future changes in the 0.8 branch, we double commit them to
> > trunk.
> > > > Any
> > > > > > objections?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > >
> > > > > > On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com>
> wrote:
> > > > > >
> > > > > > > Hi, Everyone,
> > > > > > >
> > > > > > > I made another pass of the remaining jiras that we plan to fix
> in
> > > the
> > > > > 0.8
> > > > > > > final release.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > > > > >
> > > > > > > Do people agree with this list?
> > > > > > >
> > > > > > > Joe,
> > > > > > >
> > > > > > > I don't have good understanding of KAFKA-1018. Do you think
> this
> > > > needs
> > > > > to
> > > > > > > be fixed in 0.8 final?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com>
> > wrote:
> > > > > > >
> > > > > > >> Hi, Everyone,
> > > > > > >>
> > > > > > >> We have been stabilizing the 0.8 branch since the beta1
> > release. I
> > > > > think
> > > > > > >> we are getting close to an 0.8 final release. I made an
> initial
> > > list
> > > > > of
> > > > > > the
> > > > > > >> remaining jiras that should be fixed in 0.8.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > > > > >>
> > > > > > >> 1. Do people agree with the list?
> > > > > > >>
> > > > > > >> 2. If the list is good, could people help
> contributing/reviewing
> > > the
> > > > > > >> remaining jiras?
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >>
> > > > > > >> Jun
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: preparing for the 0.8 final release

Posted by Neha Narkhede <ne...@gmail.com>.
Can you send around a thread dump of the halted consumer process?



On Sat, Oct 19, 2013 at 12:16 PM, Jason Rosenberg <jb...@squareup.com> wrote:

> The latest HEAD does seem to solve one issue, where a new topic being
> created after the consumer is started, would not be consumed.
>
> But the bigger issue is that we have a couple different consumers both
> consuming the same set of topics (under different groupids), and hanging
> after a while (both hanging at about the same point).  The topics in each
> case are selected with a filter (actually a relatively large number of
> topics, some of which are newly created over time).  I'm still not sure
> whether the new version is solving this issue (since it was a rare
> transient thing anyway).
>
> Thanks,
>
> Jason
>
>
> On Sat, Oct 19, 2013 at 2:03 AM, Jun Rao <ju...@gmail.com> wrote:
>
> > Yes, 0.8 will be released from the HEAD of the 0.8 branch. Is the problem
> > with consuming new topics or topics whose partitions are increased? If
> so,
> > see KAFKA-1030 and KAFKA-1075.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Fri, Oct 18, 2013 at 4:03 PM, Jason Rosenberg <jb...@squareup.com>
> wrote:
> >
> > > Will the 0.8 release come from the HEAD of the 0.8 branch?  I'd like to
> > > experiment with it, to see if it solves some of the issues I'm seeing,
> > with
> > > consumers refusing to consume new messages.  We've been using the beta1
> > > version.
> > >
> > > I remember mention there was a Jira issues along these lines, which was
> > > fixed post 0.8-beta1.  Which issue was that (I'd like to see if it
> > matches
> > > what I'm seeing).
> > >
> > > Thanks,
> > >
> > > Jason
> > >
> > >
> > > On Wed, Oct 9, 2013 at 8:04 PM, Jay Kreps <ja...@gmail.com> wrote:
> > >
> > > > I uploaded a patch against trunk which also fixes KAFKA-1036, the
> other
> > > > knows windows issue. Review appreciated. Should be an easy one.
> > > >
> > > > https://issues.apache.org/jira/browse/KAFKA-1008
> > > >
> > > > -Jay
> > > >
> > > >
> > > > On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:
> > > >
> > > > > KAFKA-1008 has been checked into the 0.8 branch and needs to be
> > > manually
> > > > > double-committed to trunk. To avoid merging problems, I suggest
> that
> > > for
> > > > > all future changes in the 0.8 branch, we double commit them to
> trunk.
> > > Any
> > > > > objections?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > >
> > > > > On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
> > > > >
> > > > > > Hi, Everyone,
> > > > > >
> > > > > > I made another pass of the remaining jiras that we plan to fix in
> > the
> > > > 0.8
> > > > > > final release.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > > > >
> > > > > > Do people agree with this list?
> > > > > >
> > > > > > Joe,
> > > > > >
> > > > > > I don't have good understanding of KAFKA-1018. Do you think this
> > > needs
> > > > to
> > > > > > be fixed in 0.8 final?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com>
> wrote:
> > > > > >
> > > > > >> Hi, Everyone,
> > > > > >>
> > > > > >> We have been stabilizing the 0.8 branch since the beta1
> release. I
> > > > think
> > > > > >> we are getting close to an 0.8 final release. I made an initial
> > list
> > > > of
> > > > > the
> > > > > >> remaining jiras that should be fixed in 0.8.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > > > >>
> > > > > >> 1. Do people agree with the list?
> > > > > >>
> > > > > >> 2. If the list is good, could people help contributing/reviewing
> > the
> > > > > >> remaining jiras?
> > > > > >>
> > > > > >> Thanks,
> > > > > >>
> > > > > >> Jun
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: preparing for the 0.8 final release

Posted by Neha Narkhede <ne...@gmail.com>.
Can you send around a thread dump of the halted consumer process?



On Sat, Oct 19, 2013 at 12:16 PM, Jason Rosenberg <jb...@squareup.com> wrote:

> The latest HEAD does seem to solve one issue, where a new topic being
> created after the consumer is started, would not be consumed.
>
> But the bigger issue is that we have a couple different consumers both
> consuming the same set of topics (under different groupids), and hanging
> after a while (both hanging at about the same point).  The topics in each
> case are selected with a filter (actually a relatively large number of
> topics, some of which are newly created over time).  I'm still not sure
> whether the new version is solving this issue (since it was a rare
> transient thing anyway).
>
> Thanks,
>
> Jason
>
>
> On Sat, Oct 19, 2013 at 2:03 AM, Jun Rao <ju...@gmail.com> wrote:
>
> > Yes, 0.8 will be released from the HEAD of the 0.8 branch. Is the problem
> > with consuming new topics or topics whose partitions are increased? If
> so,
> > see KAFKA-1030 and KAFKA-1075.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Fri, Oct 18, 2013 at 4:03 PM, Jason Rosenberg <jb...@squareup.com>
> wrote:
> >
> > > Will the 0.8 release come from the HEAD of the 0.8 branch?  I'd like to
> > > experiment with it, to see if it solves some of the issues I'm seeing,
> > with
> > > consumers refusing to consume new messages.  We've been using the beta1
> > > version.
> > >
> > > I remember mention there was a Jira issues along these lines, which was
> > > fixed post 0.8-beta1.  Which issue was that (I'd like to see if it
> > matches
> > > what I'm seeing).
> > >
> > > Thanks,
> > >
> > > Jason
> > >
> > >
> > > On Wed, Oct 9, 2013 at 8:04 PM, Jay Kreps <ja...@gmail.com> wrote:
> > >
> > > > I uploaded a patch against trunk which also fixes KAFKA-1036, the
> other
> > > > knows windows issue. Review appreciated. Should be an easy one.
> > > >
> > > > https://issues.apache.org/jira/browse/KAFKA-1008
> > > >
> > > > -Jay
> > > >
> > > >
> > > > On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:
> > > >
> > > > > KAFKA-1008 has been checked into the 0.8 branch and needs to be
> > > manually
> > > > > double-committed to trunk. To avoid merging problems, I suggest
> that
> > > for
> > > > > all future changes in the 0.8 branch, we double commit them to
> trunk.
> > > Any
> > > > > objections?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > >
> > > > > On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
> > > > >
> > > > > > Hi, Everyone,
> > > > > >
> > > > > > I made another pass of the remaining jiras that we plan to fix in
> > the
> > > > 0.8
> > > > > > final release.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > > > >
> > > > > > Do people agree with this list?
> > > > > >
> > > > > > Joe,
> > > > > >
> > > > > > I don't have good understanding of KAFKA-1018. Do you think this
> > > needs
> > > > to
> > > > > > be fixed in 0.8 final?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com>
> wrote:
> > > > > >
> > > > > >> Hi, Everyone,
> > > > > >>
> > > > > >> We have been stabilizing the 0.8 branch since the beta1
> release. I
> > > > think
> > > > > >> we are getting close to an 0.8 final release. I made an initial
> > list
> > > > of
> > > > > the
> > > > > >> remaining jiras that should be fixed in 0.8.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > > > >>
> > > > > >> 1. Do people agree with the list?
> > > > > >>
> > > > > >> 2. If the list is good, could people help contributing/reviewing
> > the
> > > > > >> remaining jiras?
> > > > > >>
> > > > > >> Thanks,
> > > > > >>
> > > > > >> Jun
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: preparing for the 0.8 final release

Posted by Jason Rosenberg <jb...@squareup.com>.
The latest HEAD does seem to solve one issue, where a new topic being
created after the consumer is started, would not be consumed.

But the bigger issue is that we have a couple different consumers both
consuming the same set of topics (under different groupids), and hanging
after a while (both hanging at about the same point).  The topics in each
case are selected with a filter (actually a relatively large number of
topics, some of which are newly created over time).  I'm still not sure
whether the new version is solving this issue (since it was a rare
transient thing anyway).

Thanks,

Jason


On Sat, Oct 19, 2013 at 2:03 AM, Jun Rao <ju...@gmail.com> wrote:

> Yes, 0.8 will be released from the HEAD of the 0.8 branch. Is the problem
> with consuming new topics or topics whose partitions are increased? If so,
> see KAFKA-1030 and KAFKA-1075.
>
> Thanks,
>
> Jun
>
>
> On Fri, Oct 18, 2013 at 4:03 PM, Jason Rosenberg <jb...@squareup.com> wrote:
>
> > Will the 0.8 release come from the HEAD of the 0.8 branch?  I'd like to
> > experiment with it, to see if it solves some of the issues I'm seeing,
> with
> > consumers refusing to consume new messages.  We've been using the beta1
> > version.
> >
> > I remember mention there was a Jira issues along these lines, which was
> > fixed post 0.8-beta1.  Which issue was that (I'd like to see if it
> matches
> > what I'm seeing).
> >
> > Thanks,
> >
> > Jason
> >
> >
> > On Wed, Oct 9, 2013 at 8:04 PM, Jay Kreps <ja...@gmail.com> wrote:
> >
> > > I uploaded a patch against trunk which also fixes KAFKA-1036, the other
> > > knows windows issue. Review appreciated. Should be an easy one.
> > >
> > > https://issues.apache.org/jira/browse/KAFKA-1008
> > >
> > > -Jay
> > >
> > >
> > > On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:
> > >
> > > > KAFKA-1008 has been checked into the 0.8 branch and needs to be
> > manually
> > > > double-committed to trunk. To avoid merging problems, I suggest that
> > for
> > > > all future changes in the 0.8 branch, we double commit them to trunk.
> > Any
> > > > objections?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > >
> > > > On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
> > > >
> > > > > Hi, Everyone,
> > > > >
> > > > > I made another pass of the remaining jiras that we plan to fix in
> the
> > > 0.8
> > > > > final release.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > > >
> > > > > Do people agree with this list?
> > > > >
> > > > > Joe,
> > > > >
> > > > > I don't have good understanding of KAFKA-1018. Do you think this
> > needs
> > > to
> > > > > be fixed in 0.8 final?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
> > > > >
> > > > >> Hi, Everyone,
> > > > >>
> > > > >> We have been stabilizing the 0.8 branch since the beta1 release. I
> > > think
> > > > >> we are getting close to an 0.8 final release. I made an initial
> list
> > > of
> > > > the
> > > > >> remaining jiras that should be fixed in 0.8.
> > > > >>
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > > >>
> > > > >> 1. Do people agree with the list?
> > > > >>
> > > > >> 2. If the list is good, could people help contributing/reviewing
> the
> > > > >> remaining jiras?
> > > > >>
> > > > >> Thanks,
> > > > >>
> > > > >> Jun
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: preparing for the 0.8 final release

Posted by Jason Rosenberg <jb...@squareup.com>.
The latest HEAD does seem to solve one issue, where a new topic being
created after the consumer is started, would not be consumed.

But the bigger issue is that we have a couple different consumers both
consuming the same set of topics (under different groupids), and hanging
after a while (both hanging at about the same point).  The topics in each
case are selected with a filter (actually a relatively large number of
topics, some of which are newly created over time).  I'm still not sure
whether the new version is solving this issue (since it was a rare
transient thing anyway).

Thanks,

Jason


On Sat, Oct 19, 2013 at 2:03 AM, Jun Rao <ju...@gmail.com> wrote:

> Yes, 0.8 will be released from the HEAD of the 0.8 branch. Is the problem
> with consuming new topics or topics whose partitions are increased? If so,
> see KAFKA-1030 and KAFKA-1075.
>
> Thanks,
>
> Jun
>
>
> On Fri, Oct 18, 2013 at 4:03 PM, Jason Rosenberg <jb...@squareup.com> wrote:
>
> > Will the 0.8 release come from the HEAD of the 0.8 branch?  I'd like to
> > experiment with it, to see if it solves some of the issues I'm seeing,
> with
> > consumers refusing to consume new messages.  We've been using the beta1
> > version.
> >
> > I remember mention there was a Jira issues along these lines, which was
> > fixed post 0.8-beta1.  Which issue was that (I'd like to see if it
> matches
> > what I'm seeing).
> >
> > Thanks,
> >
> > Jason
> >
> >
> > On Wed, Oct 9, 2013 at 8:04 PM, Jay Kreps <ja...@gmail.com> wrote:
> >
> > > I uploaded a patch against trunk which also fixes KAFKA-1036, the other
> > > knows windows issue. Review appreciated. Should be an easy one.
> > >
> > > https://issues.apache.org/jira/browse/KAFKA-1008
> > >
> > > -Jay
> > >
> > >
> > > On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:
> > >
> > > > KAFKA-1008 has been checked into the 0.8 branch and needs to be
> > manually
> > > > double-committed to trunk. To avoid merging problems, I suggest that
> > for
> > > > all future changes in the 0.8 branch, we double commit them to trunk.
> > Any
> > > > objections?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > >
> > > > On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
> > > >
> > > > > Hi, Everyone,
> > > > >
> > > > > I made another pass of the remaining jiras that we plan to fix in
> the
> > > 0.8
> > > > > final release.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > > >
> > > > > Do people agree with this list?
> > > > >
> > > > > Joe,
> > > > >
> > > > > I don't have good understanding of KAFKA-1018. Do you think this
> > needs
> > > to
> > > > > be fixed in 0.8 final?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
> > > > >
> > > > >> Hi, Everyone,
> > > > >>
> > > > >> We have been stabilizing the 0.8 branch since the beta1 release. I
> > > think
> > > > >> we are getting close to an 0.8 final release. I made an initial
> list
> > > of
> > > > the
> > > > >> remaining jiras that should be fixed in 0.8.
> > > > >>
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > > >>
> > > > >> 1. Do people agree with the list?
> > > > >>
> > > > >> 2. If the list is good, could people help contributing/reviewing
> the
> > > > >> remaining jiras?
> > > > >>
> > > > >> Thanks,
> > > > >>
> > > > >> Jun
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: preparing for the 0.8 final release

Posted by Jun Rao <ju...@gmail.com>.
Yes, 0.8 will be released from the HEAD of the 0.8 branch. Is the problem
with consuming new topics or topics whose partitions are increased? If so,
see KAFKA-1030 and KAFKA-1075.

Thanks,

Jun


On Fri, Oct 18, 2013 at 4:03 PM, Jason Rosenberg <jb...@squareup.com> wrote:

> Will the 0.8 release come from the HEAD of the 0.8 branch?  I'd like to
> experiment with it, to see if it solves some of the issues I'm seeing, with
> consumers refusing to consume new messages.  We've been using the beta1
> version.
>
> I remember mention there was a Jira issues along these lines, which was
> fixed post 0.8-beta1.  Which issue was that (I'd like to see if it matches
> what I'm seeing).
>
> Thanks,
>
> Jason
>
>
> On Wed, Oct 9, 2013 at 8:04 PM, Jay Kreps <ja...@gmail.com> wrote:
>
> > I uploaded a patch against trunk which also fixes KAFKA-1036, the other
> > knows windows issue. Review appreciated. Should be an easy one.
> >
> > https://issues.apache.org/jira/browse/KAFKA-1008
> >
> > -Jay
> >
> >
> > On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> > > KAFKA-1008 has been checked into the 0.8 branch and needs to be
> manually
> > > double-committed to trunk. To avoid merging problems, I suggest that
> for
> > > all future changes in the 0.8 branch, we double commit them to trunk.
> Any
> > > objections?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
> > >
> > > > Hi, Everyone,
> > > >
> > > > I made another pass of the remaining jiras that we plan to fix in the
> > 0.8
> > > > final release.
> > > >
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > >
> > > > Do people agree with this list?
> > > >
> > > > Joe,
> > > >
> > > > I don't have good understanding of KAFKA-1018. Do you think this
> needs
> > to
> > > > be fixed in 0.8 final?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > >
> > > >
> > > > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
> > > >
> > > >> Hi, Everyone,
> > > >>
> > > >> We have been stabilizing the 0.8 branch since the beta1 release. I
> > think
> > > >> we are getting close to an 0.8 final release. I made an initial list
> > of
> > > the
> > > >> remaining jiras that should be fixed in 0.8.
> > > >>
> > > >>
> > > >>
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > >>
> > > >> 1. Do people agree with the list?
> > > >>
> > > >> 2. If the list is good, could people help contributing/reviewing the
> > > >> remaining jiras?
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Jun
> > > >>
> > > >
> > > >
> > >
> >
>

Re: preparing for the 0.8 final release

Posted by Jun Rao <ju...@gmail.com>.
Yes, 0.8 will be released from the HEAD of the 0.8 branch. Is the problem
with consuming new topics or topics whose partitions are increased? If so,
see KAFKA-1030 and KAFKA-1075.

Thanks,

Jun


On Fri, Oct 18, 2013 at 4:03 PM, Jason Rosenberg <jb...@squareup.com> wrote:

> Will the 0.8 release come from the HEAD of the 0.8 branch?  I'd like to
> experiment with it, to see if it solves some of the issues I'm seeing, with
> consumers refusing to consume new messages.  We've been using the beta1
> version.
>
> I remember mention there was a Jira issues along these lines, which was
> fixed post 0.8-beta1.  Which issue was that (I'd like to see if it matches
> what I'm seeing).
>
> Thanks,
>
> Jason
>
>
> On Wed, Oct 9, 2013 at 8:04 PM, Jay Kreps <ja...@gmail.com> wrote:
>
> > I uploaded a patch against trunk which also fixes KAFKA-1036, the other
> > knows windows issue. Review appreciated. Should be an easy one.
> >
> > https://issues.apache.org/jira/browse/KAFKA-1008
> >
> > -Jay
> >
> >
> > On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> > > KAFKA-1008 has been checked into the 0.8 branch and needs to be
> manually
> > > double-committed to trunk. To avoid merging problems, I suggest that
> for
> > > all future changes in the 0.8 branch, we double commit them to trunk.
> Any
> > > objections?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
> > >
> > > > Hi, Everyone,
> > > >
> > > > I made another pass of the remaining jiras that we plan to fix in the
> > 0.8
> > > > final release.
> > > >
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > >
> > > > Do people agree with this list?
> > > >
> > > > Joe,
> > > >
> > > > I don't have good understanding of KAFKA-1018. Do you think this
> needs
> > to
> > > > be fixed in 0.8 final?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > >
> > > >
> > > > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
> > > >
> > > >> Hi, Everyone,
> > > >>
> > > >> We have been stabilizing the 0.8 branch since the beta1 release. I
> > think
> > > >> we are getting close to an 0.8 final release. I made an initial list
> > of
> > > the
> > > >> remaining jiras that should be fixed in 0.8.
> > > >>
> > > >>
> > > >>
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > > >>
> > > >> 1. Do people agree with the list?
> > > >>
> > > >> 2. If the list is good, could people help contributing/reviewing the
> > > >> remaining jiras?
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Jun
> > > >>
> > > >
> > > >
> > >
> >
>

Re: preparing for the 0.8 final release

Posted by Jason Rosenberg <jb...@squareup.com>.
Will the 0.8 release come from the HEAD of the 0.8 branch?  I'd like to
experiment with it, to see if it solves some of the issues I'm seeing, with
consumers refusing to consume new messages.  We've been using the beta1
version.

I remember mention there was a Jira issues along these lines, which was
fixed post 0.8-beta1.  Which issue was that (I'd like to see if it matches
what I'm seeing).

Thanks,

Jason


On Wed, Oct 9, 2013 at 8:04 PM, Jay Kreps <ja...@gmail.com> wrote:

> I uploaded a patch against trunk which also fixes KAFKA-1036, the other
> knows windows issue. Review appreciated. Should be an easy one.
>
> https://issues.apache.org/jira/browse/KAFKA-1008
>
> -Jay
>
>
> On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:
>
> > KAFKA-1008 has been checked into the 0.8 branch and needs to be manually
> > double-committed to trunk. To avoid merging problems, I suggest that for
> > all future changes in the 0.8 branch, we double commit them to trunk. Any
> > objections?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
> >
> > > Hi, Everyone,
> > >
> > > I made another pass of the remaining jiras that we plan to fix in the
> 0.8
> > > final release.
> > >
> > >
> > >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > >
> > > Do people agree with this list?
> > >
> > > Joe,
> > >
> > > I don't have good understanding of KAFKA-1018. Do you think this needs
> to
> > > be fixed in 0.8 final?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > >
> > > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
> > >
> > >> Hi, Everyone,
> > >>
> > >> We have been stabilizing the 0.8 branch since the beta1 release. I
> think
> > >> we are getting close to an 0.8 final release. I made an initial list
> of
> > the
> > >> remaining jiras that should be fixed in 0.8.
> > >>
> > >>
> > >>
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > >>
> > >> 1. Do people agree with the list?
> > >>
> > >> 2. If the list is good, could people help contributing/reviewing the
> > >> remaining jiras?
> > >>
> > >> Thanks,
> > >>
> > >> Jun
> > >>
> > >
> > >
> >
>

Re: preparing for the 0.8 final release

Posted by Jason Rosenberg <jb...@squareup.com>.
Will the 0.8 release come from the HEAD of the 0.8 branch?  I'd like to
experiment with it, to see if it solves some of the issues I'm seeing, with
consumers refusing to consume new messages.  We've been using the beta1
version.

I remember mention there was a Jira issues along these lines, which was
fixed post 0.8-beta1.  Which issue was that (I'd like to see if it matches
what I'm seeing).

Thanks,

Jason


On Wed, Oct 9, 2013 at 8:04 PM, Jay Kreps <ja...@gmail.com> wrote:

> I uploaded a patch against trunk which also fixes KAFKA-1036, the other
> knows windows issue. Review appreciated. Should be an easy one.
>
> https://issues.apache.org/jira/browse/KAFKA-1008
>
> -Jay
>
>
> On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:
>
> > KAFKA-1008 has been checked into the 0.8 branch and needs to be manually
> > double-committed to trunk. To avoid merging problems, I suggest that for
> > all future changes in the 0.8 branch, we double commit them to trunk. Any
> > objections?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
> >
> > > Hi, Everyone,
> > >
> > > I made another pass of the remaining jiras that we plan to fix in the
> 0.8
> > > final release.
> > >
> > >
> > >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > >
> > > Do people agree with this list?
> > >
> > > Joe,
> > >
> > > I don't have good understanding of KAFKA-1018. Do you think this needs
> to
> > > be fixed in 0.8 final?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > >
> > > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
> > >
> > >> Hi, Everyone,
> > >>
> > >> We have been stabilizing the 0.8 branch since the beta1 release. I
> think
> > >> we are getting close to an 0.8 final release. I made an initial list
> of
> > the
> > >> remaining jiras that should be fixed in 0.8.
> > >>
> > >>
> > >>
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> > >>
> > >> 1. Do people agree with the list?
> > >>
> > >> 2. If the list is good, could people help contributing/reviewing the
> > >> remaining jiras?
> > >>
> > >> Thanks,
> > >>
> > >> Jun
> > >>
> > >
> > >
> >
>

Re: preparing for the 0.8 final release

Posted by Jay Kreps <ja...@gmail.com>.
I uploaded a patch against trunk which also fixes KAFKA-1036, the other
knows windows issue. Review appreciated. Should be an easy one.

https://issues.apache.org/jira/browse/KAFKA-1008

-Jay


On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:

> KAFKA-1008 has been checked into the 0.8 branch and needs to be manually
> double-committed to trunk. To avoid merging problems, I suggest that for
> all future changes in the 0.8 branch, we double commit them to trunk. Any
> objections?
>
> Thanks,
>
> Jun
>
>
> On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
>
> > Hi, Everyone,
> >
> > I made another pass of the remaining jiras that we plan to fix in the 0.8
> > final release.
> >
> >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >
> > Do people agree with this list?
> >
> > Joe,
> >
> > I don't have good understanding of KAFKA-1018. Do you think this needs to
> > be fixed in 0.8 final?
> >
> > Thanks,
> >
> > Jun
> >
> >
> >
> > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> >> Hi, Everyone,
> >>
> >> We have been stabilizing the 0.8 branch since the beta1 release. I think
> >> we are getting close to an 0.8 final release. I made an initial list of
> the
> >> remaining jiras that should be fixed in 0.8.
> >>
> >>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >>
> >> 1. Do people agree with the list?
> >>
> >> 2. If the list is good, could people help contributing/reviewing the
> >> remaining jiras?
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >
> >
>

Re: preparing for the 0.8 final release

Posted by Neha Narkhede <ne...@gmail.com>.
Sounds good to me.

Thanks,
Neha


On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:

> KAFKA-1008 has been checked into the 0.8 branch and needs to be manually
> double-committed to trunk. To avoid merging problems, I suggest that for
> all future changes in the 0.8 branch, we double commit them to trunk. Any
> objections?
>
> Thanks,
>
> Jun
>
>
> On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
>
> > Hi, Everyone,
> >
> > I made another pass of the remaining jiras that we plan to fix in the 0.8
> > final release.
> >
> >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >
> > Do people agree with this list?
> >
> > Joe,
> >
> > I don't have good understanding of KAFKA-1018. Do you think this needs to
> > be fixed in 0.8 final?
> >
> > Thanks,
> >
> > Jun
> >
> >
> >
> > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> >> Hi, Everyone,
> >>
> >> We have been stabilizing the 0.8 branch since the beta1 release. I think
> >> we are getting close to an 0.8 final release. I made an initial list of
> the
> >> remaining jiras that should be fixed in 0.8.
> >>
> >>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >>
> >> 1. Do people agree with the list?
> >>
> >> 2. If the list is good, could people help contributing/reviewing the
> >> remaining jiras?
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >
> >
>

Re: preparing for the 0.8 final release

Posted by Jay Kreps <ja...@gmail.com>.
I uploaded a patch against trunk which also fixes KAFKA-1036, the other
knows windows issue. Review appreciated. Should be an easy one.

https://issues.apache.org/jira/browse/KAFKA-1008

-Jay


On Wed, Oct 9, 2013 at 8:56 AM, Jun Rao <ju...@gmail.com> wrote:

> KAFKA-1008 has been checked into the 0.8 branch and needs to be manually
> double-committed to trunk. To avoid merging problems, I suggest that for
> all future changes in the 0.8 branch, we double commit them to trunk. Any
> objections?
>
> Thanks,
>
> Jun
>
>
> On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
>
> > Hi, Everyone,
> >
> > I made another pass of the remaining jiras that we plan to fix in the 0.8
> > final release.
> >
> >
> >
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >
> > Do people agree with this list?
> >
> > Joe,
> >
> > I don't have good understanding of KAFKA-1018. Do you think this needs to
> > be fixed in 0.8 final?
> >
> > Thanks,
> >
> > Jun
> >
> >
> >
> > On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> >> Hi, Everyone,
> >>
> >> We have been stabilizing the 0.8 branch since the beta1 release. I think
> >> we are getting close to an 0.8 final release. I made an initial list of
> the
> >> remaining jiras that should be fixed in 0.8.
> >>
> >>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
> >>
> >> 1. Do people agree with the list?
> >>
> >> 2. If the list is good, could people help contributing/reviewing the
> >> remaining jiras?
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >
> >
>

Re: preparing for the 0.8 final release

Posted by Jun Rao <ju...@gmail.com>.
KAFKA-1008 has been checked into the 0.8 branch and needs to be manually
double-committed to trunk. To avoid merging problems, I suggest that for
all future changes in the 0.8 branch, we double commit them to trunk. Any
objections?

Thanks,

Jun


On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:

> Hi, Everyone,
>
> I made another pass of the remaining jiras that we plan to fix in the 0.8
> final release.
>
>
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>
> Do people agree with this list?
>
> Joe,
>
> I don't have good understanding of KAFKA-1018. Do you think this needs to
> be fixed in 0.8 final?
>
> Thanks,
>
> Jun
>
>
>
> On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
>
>> Hi, Everyone,
>>
>> We have been stabilizing the 0.8 branch since the beta1 release. I think
>> we are getting close to an 0.8 final release. I made an initial list of the
>> remaining jiras that should be fixed in 0.8.
>>
>>
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>>
>> 1. Do people agree with the list?
>>
>> 2. If the list is good, could people help contributing/reviewing the
>> remaining jiras?
>>
>> Thanks,
>>
>> Jun
>>
>
>

Re: preparing for the 0.8 final release

Posted by Jun Rao <ju...@gmail.com>.
KAFKA-1008 has been checked into the 0.8 branch and needs to be manually
double-committed to trunk. To avoid merging problems, I suggest that for
all future changes in the 0.8 branch, we double commit them to trunk. Any
objections?

Thanks,

Jun


On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:

> Hi, Everyone,
>
> I made another pass of the remaining jiras that we plan to fix in the 0.8
> final release.
>
>
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>
> Do people agree with this list?
>
> Joe,
>
> I don't have good understanding of KAFKA-1018. Do you think this needs to
> be fixed in 0.8 final?
>
> Thanks,
>
> Jun
>
>
>
> On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
>
>> Hi, Everyone,
>>
>> We have been stabilizing the 0.8 branch since the beta1 release. I think
>> we are getting close to an 0.8 final release. I made an initial list of the
>> remaining jiras that should be fixed in 0.8.
>>
>>
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>>
>> 1. Do people agree with the list?
>>
>> 2. If the list is good, could people help contributing/reviewing the
>> remaining jiras?
>>
>> Thanks,
>>
>> Jun
>>
>
>

Re: preparing for the 0.8 final release

Posted by Joe Stein <jo...@stealth.ly>.
Sounds good, yup!


/*******************************************
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop
********************************************/


On Oct 24, 2013, at 1:12 PM, Jun Rao <ju...@gmail.com> wrote:

> At this moment, we have resolved all jiras that we intend to fix in 0.8.0
> final.
> 
> Joe,
> 
> Would you like to drive the 0.8.0 final release again?
> 
> Thanks,
> 
> Jun
> 
> 
> On Mon, Oct 21, 2013 at 8:53 PM, Jun Rao <ju...@gmail.com> wrote:
> 
>> Hi, Everyone,
>> 
>> At this moment, we have only one remaining jira (KAFKA-1097) that we plan
>> to fix in 0.8. After that, we can cut the final 0.8 release.
>> 
>> Thanks,
>> 
>> Jun
>> 
>> 
>> 
>> On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
>> 
>>> Hi, Everyone,
>>> 
>>> I made another pass of the remaining jiras that we plan to fix in the 0.8
>>> final release.
>>> 
>>> 
>>> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>>> 
>>> Do people agree with this list?
>>> 
>>> Joe,
>>> 
>>> I don't have good understanding of KAFKA-1018. Do you think this needs to
>>> be fixed in 0.8 final?
>>> 
>>> Thanks,
>>> 
>>> Jun
>>> 
>>> 
>>> 
>>> On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
>>> 
>>>> Hi, Everyone,
>>>> 
>>>> We have been stabilizing the 0.8 branch since the beta1 release. I think
>>>> we are getting close to an 0.8 final release. I made an initial list of the
>>>> remaining jiras that should be fixed in 0.8.
>>>> 
>>>> 
>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>>>> 
>>>> 1. Do people agree with the list?
>>>> 
>>>> 2. If the list is good, could people help contributing/reviewing the
>>>> remaining jiras?
>>>> 
>>>> Thanks,
>>>> 
>>>> Jun
>>>> 
>>> 
>>> 
>>

Re: preparing for the 0.8 final release

Posted by Joe Stein <jo...@stealth.ly>.
Sounds good, yup!


/*******************************************
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop
********************************************/


On Oct 24, 2013, at 1:12 PM, Jun Rao <ju...@gmail.com> wrote:

> At this moment, we have resolved all jiras that we intend to fix in 0.8.0
> final.
> 
> Joe,
> 
> Would you like to drive the 0.8.0 final release again?
> 
> Thanks,
> 
> Jun
> 
> 
> On Mon, Oct 21, 2013 at 8:53 PM, Jun Rao <ju...@gmail.com> wrote:
> 
>> Hi, Everyone,
>> 
>> At this moment, we have only one remaining jira (KAFKA-1097) that we plan
>> to fix in 0.8. After that, we can cut the final 0.8 release.
>> 
>> Thanks,
>> 
>> Jun
>> 
>> 
>> 
>> On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
>> 
>>> Hi, Everyone,
>>> 
>>> I made another pass of the remaining jiras that we plan to fix in the 0.8
>>> final release.
>>> 
>>> 
>>> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>>> 
>>> Do people agree with this list?
>>> 
>>> Joe,
>>> 
>>> I don't have good understanding of KAFKA-1018. Do you think this needs to
>>> be fixed in 0.8 final?
>>> 
>>> Thanks,
>>> 
>>> Jun
>>> 
>>> 
>>> 
>>> On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
>>> 
>>>> Hi, Everyone,
>>>> 
>>>> We have been stabilizing the 0.8 branch since the beta1 release. I think
>>>> we are getting close to an 0.8 final release. I made an initial list of the
>>>> remaining jiras that should be fixed in 0.8.
>>>> 
>>>> 
>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>>>> 
>>>> 1. Do people agree with the list?
>>>> 
>>>> 2. If the list is good, could people help contributing/reviewing the
>>>> remaining jiras?
>>>> 
>>>> Thanks,
>>>> 
>>>> Jun
>>>> 
>>> 
>>> 
>>

Re: preparing for the 0.8 final release

Posted by Jun Rao <ju...@gmail.com>.
At this moment, we have resolved all jiras that we intend to fix in 0.8.0
final.

Joe,

Would you like to drive the 0.8.0 final release again?

Thanks,

Jun


On Mon, Oct 21, 2013 at 8:53 PM, Jun Rao <ju...@gmail.com> wrote:

> Hi, Everyone,
>
> At this moment, we have only one remaining jira (KAFKA-1097) that we plan
> to fix in 0.8. After that, we can cut the final 0.8 release.
>
> Thanks,
>
> Jun
>
>
>
> On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
>
>> Hi, Everyone,
>>
>> I made another pass of the remaining jiras that we plan to fix in the 0.8
>> final release.
>>
>>
>> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>>
>> Do people agree with this list?
>>
>> Joe,
>>
>> I don't have good understanding of KAFKA-1018. Do you think this needs to
>> be fixed in 0.8 final?
>>
>> Thanks,
>>
>> Jun
>>
>>
>>
>> On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
>>
>>> Hi, Everyone,
>>>
>>> We have been stabilizing the 0.8 branch since the beta1 release. I think
>>> we are getting close to an 0.8 final release. I made an initial list of the
>>> remaining jiras that should be fixed in 0.8.
>>>
>>>
>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>>>
>>> 1. Do people agree with the list?
>>>
>>> 2. If the list is good, could people help contributing/reviewing the
>>> remaining jiras?
>>>
>>> Thanks,
>>>
>>> Jun
>>>
>>
>>
>

Re: preparing for the 0.8 final release

Posted by Jun Rao <ju...@gmail.com>.
At this moment, we have resolved all jiras that we intend to fix in 0.8.0
final.

Joe,

Would you like to drive the 0.8.0 final release again?

Thanks,

Jun


On Mon, Oct 21, 2013 at 8:53 PM, Jun Rao <ju...@gmail.com> wrote:

> Hi, Everyone,
>
> At this moment, we have only one remaining jira (KAFKA-1097) that we plan
> to fix in 0.8. After that, we can cut the final 0.8 release.
>
> Thanks,
>
> Jun
>
>
>
> On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:
>
>> Hi, Everyone,
>>
>> I made another pass of the remaining jiras that we plan to fix in the 0.8
>> final release.
>>
>>
>> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>>
>> Do people agree with this list?
>>
>> Joe,
>>
>> I don't have good understanding of KAFKA-1018. Do you think this needs to
>> be fixed in 0.8 final?
>>
>> Thanks,
>>
>> Jun
>>
>>
>>
>> On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
>>
>>> Hi, Everyone,
>>>
>>> We have been stabilizing the 0.8 branch since the beta1 release. I think
>>> we are getting close to an 0.8 final release. I made an initial list of the
>>> remaining jiras that should be fixed in 0.8.
>>>
>>>
>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>>>
>>> 1. Do people agree with the list?
>>>
>>> 2. If the list is good, could people help contributing/reviewing the
>>> remaining jiras?
>>>
>>> Thanks,
>>>
>>> Jun
>>>
>>
>>
>

Re: preparing for the 0.8 final release

Posted by Jun Rao <ju...@gmail.com>.
Hi, Everyone,

At this moment, we have only one remaining jira (KAFKA-1097) that we plan
to fix in 0.8. After that, we can cut the final 0.8 release.

Thanks,

Jun


On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:

> Hi, Everyone,
>
> I made another pass of the remaining jiras that we plan to fix in the 0.8
> final release.
>
>
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>
> Do people agree with this list?
>
> Joe,
>
> I don't have good understanding of KAFKA-1018. Do you think this needs to
> be fixed in 0.8 final?
>
> Thanks,
>
> Jun
>
>
>
> On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
>
>> Hi, Everyone,
>>
>> We have been stabilizing the 0.8 branch since the beta1 release. I think
>> we are getting close to an 0.8 final release. I made an initial list of the
>> remaining jiras that should be fixed in 0.8.
>>
>>
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>>
>> 1. Do people agree with the list?
>>
>> 2. If the list is good, could people help contributing/reviewing the
>> remaining jiras?
>>
>> Thanks,
>>
>> Jun
>>
>
>

Re: preparing for the 0.8 final release

Posted by Joe Stein <cr...@gmail.com>.
Jun,

KAFKA-1018 should be fixed in 0.8 final I can post a patch or review.

+1 on the list for whats left

On Mon, Oct 7, 2013 at 8:33 PM, Jun Rao <ju...@gmail.com> wrote:

> Hi, Everyone,
>
> I made another pass of the remaining jiras that we plan to fix in the 0.8
> final release.
>
>
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>
> Do people agree with this list?
>
> Joe,
>
> I don't have good understanding of KAFKA-1018. Do you think this needs to
> be fixed in 0.8 final?
>
> Thanks,
>
> Jun
>
>
>
> On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
>
>> Hi, Everyone,
>>
>> We have been stabilizing the 0.8 branch since the beta1 release. I think
>> we are getting close to an 0.8 final release. I made an initial list of the
>> remaining jiras that should be fixed in 0.8.
>>
>>
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>>
>> 1. Do people agree with the list?
>>
>> 2. If the list is good, could people help contributing/reviewing the
>> remaining jiras?
>>
>> Thanks,
>>
>> Jun
>>
>
>

Re: preparing for the 0.8 final release

Posted by Jun Rao <ju...@gmail.com>.
Hi, Everyone,

At this moment, we have only one remaining jira (KAFKA-1097) that we plan
to fix in 0.8. After that, we can cut the final 0.8 release.

Thanks,

Jun


On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao <ju...@gmail.com> wrote:

> Hi, Everyone,
>
> I made another pass of the remaining jiras that we plan to fix in the 0.8
> final release.
>
>
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>
> Do people agree with this list?
>
> Joe,
>
> I don't have good understanding of KAFKA-1018. Do you think this needs to
> be fixed in 0.8 final?
>
> Thanks,
>
> Jun
>
>
>
> On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
>
>> Hi, Everyone,
>>
>> We have been stabilizing the 0.8 branch since the beta1 release. I think
>> we are getting close to an 0.8 final release. I made an initial list of the
>> remaining jiras that should be fixed in 0.8.
>>
>>
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>>
>> 1. Do people agree with the list?
>>
>> 2. If the list is good, could people help contributing/reviewing the
>> remaining jiras?
>>
>> Thanks,
>>
>> Jun
>>
>
>

Re: preparing for the 0.8 final release

Posted by Joe Stein <cr...@gmail.com>.
Jun,

KAFKA-1018 should be fixed in 0.8 final I can post a patch or review.

+1 on the list for whats left

On Mon, Oct 7, 2013 at 8:33 PM, Jun Rao <ju...@gmail.com> wrote:

> Hi, Everyone,
>
> I made another pass of the remaining jiras that we plan to fix in the 0.8
> final release.
>
>
> https://issues.apache.org/jira/browse/KAFKA-954?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>
> Do people agree with this list?
>
> Joe,
>
> I don't have good understanding of KAFKA-1018. Do you think this needs to
> be fixed in 0.8 final?
>
> Thanks,
>
> Jun
>
>
>
> On Fri, Sep 13, 2013 at 9:18 AM, Jun Rao <ju...@gmail.com> wrote:
>
>> Hi, Everyone,
>>
>> We have been stabilizing the 0.8 branch since the beta1 release. I think
>> we are getting close to an 0.8 final release. I made an initial list of the
>> remaining jiras that should be fixed in 0.8.
>>
>>
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%20%220.8%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)
>>
>> 1. Do people agree with the list?
>>
>> 2. If the list is good, could people help contributing/reviewing the
>> remaining jiras?
>>
>> Thanks,
>>
>> Jun
>>
>
>