You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Jun Rao <ju...@confluent.io> on 2016/10/17 17:49:07 UTC
Kafka KIP meeting Oct 19 at 11:00am PST
Hi, Everyone.,
We plan to have a Kafka KIP meeting this coming Wednesday at 11:00am PST. If
you plan to attend but haven't received an invite, please let me know. The
following is the tentative agenda.
Agenda:
KIP-82: add record header
Thanks,
Jun
Re: Kafka KIP meeting Oct 19 at 11:00am PST
Posted by Mayuresh Gharat <gh...@gmail.com>.
I agree with Nacho.
+1 for the KIP.
Thanks,
Mayuresh
On Fri, Oct 21, 2016 at 11:46 AM, Nacho Solis <ns...@linkedin.com.invalid>
wrote:
> I think a separate KIP is a good idea as well. Note however that potential
> decisions in this KIP could affect the other KIP.
>
> Nacho
>
> On Fri, Oct 21, 2016 at 10:23 AM, Jun Rao <ju...@confluent.io> wrote:
>
> > Michael,
> >
> > Yes, doing a separate KIP to address the null payload issue for compacted
> > topics is a good idea.
> >
> > Thanks,
> >
> > Jun
> >
> > On Fri, Oct 21, 2016 at 12:57 AM, Michael Pearce <Mi...@ig.com>
> > wrote:
> >
> > > I had noted that what ever the solution having compaction based on null
> > > payload was agreed isn't elegant.
> > >
> > > Shall we raise another kip to : as discussed propose using a attribute
> > bit
> > > for delete/compaction flag as well/or instead of null value and
> updating
> > > compaction logic to look at that delelete/compaction attribute
> > >
> > > I believe this is less contentious, so that at least we get that done
> > > alleviating some concerns whilst the below gets discussed further?
> > >
> > > ________________________________________
> > > From: Jun Rao <ju...@confluent.io>
> > > Sent: Wednesday, October 19, 2016 8:56:52 PM
> > > To: dev@kafka.apache.org
> > > Subject: Re: Kafka KIP meeting Oct 19 at 11:00am PST
> > >
> > > The following are the notes from today's KIP discussion.
> > >
> > >
> > > - KIP-82 - add record header: We agreed that there are use cases for
> > > third-party vendors building tools around Kafka. We haven't reached
> > the
> > > conclusion whether the added complexity justifies the use cases. We
> > will
> > > follow up on the mailing list with use cases, container format
> people
> > > have
> > > been using, and details on the proposal.
> > >
> > >
> > > The video will be uploaded soon in https://cwiki.apache.org/
> > > confluence/display/KAFKA/Kafka+Improvement+Proposals .
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Mon, Oct 17, 2016 at 10:49 AM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Hi, Everyone.,
> > > >
> > > > We plan to have a Kafka KIP meeting this coming Wednesday at 11:00am
> > PST.
> > > > If you plan to attend but haven't received an invite, please let me
> > know.
> > > > The following is the tentative agenda.
> > > >
> > > > Agenda:
> > > > KIP-82: add record header
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > The information contained in this email is strictly confidential and
> for
> > > the use of the addressee only, unless otherwise indicated. If you are
> not
> > > the intended recipient, please do not read, copy, use or disclose to
> > others
> > > this message or any attachment. Please also notify the sender by
> replying
> > > to this email or by telephone (+44(020 7896 0011) and then delete the
> > email
> > > and any copies of it. Opinions, conclusion (etc) that do not relate to
> > the
> > > official business of this company shall be understood as neither given
> > nor
> > > endorsed by it. IG is a trading name of IG Markets Limited (a company
> > > registered in England and Wales, company number 04008957) and IG Index
> > > Limited (a company registered in England and Wales, company number
> > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
> > > London EC4R 2YA. Both IG Markets Limited (register number 195355) and
> IG
> > > Index Limited (register number 114059) are authorised and regulated by
> > the
> > > Financial Conduct Authority.
> > >
> >
>
>
>
> --
> Nacho (Ignacio) Solis
> Kafka
> nsolis@linkedin.com
>
--
-Regards,
Mayuresh R. Gharat
(862) 250-7125
Re: Kafka KIP meeting Oct 19 at 11:00am PST
Posted by Nacho Solis <ns...@linkedin.com.INVALID>.
I think a separate KIP is a good idea as well. Note however that potential
decisions in this KIP could affect the other KIP.
Nacho
On Fri, Oct 21, 2016 at 10:23 AM, Jun Rao <ju...@confluent.io> wrote:
> Michael,
>
> Yes, doing a separate KIP to address the null payload issue for compacted
> topics is a good idea.
>
> Thanks,
>
> Jun
>
> On Fri, Oct 21, 2016 at 12:57 AM, Michael Pearce <Mi...@ig.com>
> wrote:
>
> > I had noted that what ever the solution having compaction based on null
> > payload was agreed isn't elegant.
> >
> > Shall we raise another kip to : as discussed propose using a attribute
> bit
> > for delete/compaction flag as well/or instead of null value and updating
> > compaction logic to look at that delelete/compaction attribute
> >
> > I believe this is less contentious, so that at least we get that done
> > alleviating some concerns whilst the below gets discussed further?
> >
> > ________________________________________
> > From: Jun Rao <ju...@confluent.io>
> > Sent: Wednesday, October 19, 2016 8:56:52 PM
> > To: dev@kafka.apache.org
> > Subject: Re: Kafka KIP meeting Oct 19 at 11:00am PST
> >
> > The following are the notes from today's KIP discussion.
> >
> >
> > - KIP-82 - add record header: We agreed that there are use cases for
> > third-party vendors building tools around Kafka. We haven't reached
> the
> > conclusion whether the added complexity justifies the use cases. We
> will
> > follow up on the mailing list with use cases, container format people
> > have
> > been using, and details on the proposal.
> >
> >
> > The video will be uploaded soon in https://cwiki.apache.org/
> > confluence/display/KAFKA/Kafka+Improvement+Proposals .
> >
> > Thanks,
> >
> > Jun
> >
> > On Mon, Oct 17, 2016 at 10:49 AM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Everyone.,
> > >
> > > We plan to have a Kafka KIP meeting this coming Wednesday at 11:00am
> PST.
> > > If you plan to attend but haven't received an invite, please let me
> know.
> > > The following is the tentative agenda.
> > >
> > > Agenda:
> > > KIP-82: add record header
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > The information contained in this email is strictly confidential and for
> > the use of the addressee only, unless otherwise indicated. If you are not
> > the intended recipient, please do not read, copy, use or disclose to
> others
> > this message or any attachment. Please also notify the sender by replying
> > to this email or by telephone (+44(020 7896 0011) and then delete the
> email
> > and any copies of it. Opinions, conclusion (etc) that do not relate to
> the
> > official business of this company shall be understood as neither given
> nor
> > endorsed by it. IG is a trading name of IG Markets Limited (a company
> > registered in England and Wales, company number 04008957) and IG Index
> > Limited (a company registered in England and Wales, company number
> > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
> > London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG
> > Index Limited (register number 114059) are authorised and regulated by
> the
> > Financial Conduct Authority.
> >
>
--
Nacho (Ignacio) Solis
Kafka
nsolis@linkedin.com
Re: Kafka KIP meeting Oct 19 at 11:00am PST
Posted by Jun Rao <ju...@confluent.io>.
Michael,
Yes, doing a separate KIP to address the null payload issue for compacted
topics is a good idea.
Thanks,
Jun
On Fri, Oct 21, 2016 at 12:57 AM, Michael Pearce <Mi...@ig.com>
wrote:
> I had noted that what ever the solution having compaction based on null
> payload was agreed isn't elegant.
>
> Shall we raise another kip to : as discussed propose using a attribute bit
> for delete/compaction flag as well/or instead of null value and updating
> compaction logic to look at that delelete/compaction attribute
>
> I believe this is less contentious, so that at least we get that done
> alleviating some concerns whilst the below gets discussed further?
>
> ________________________________________
> From: Jun Rao <ju...@confluent.io>
> Sent: Wednesday, October 19, 2016 8:56:52 PM
> To: dev@kafka.apache.org
> Subject: Re: Kafka KIP meeting Oct 19 at 11:00am PST
>
> The following are the notes from today's KIP discussion.
>
>
> - KIP-82 - add record header: We agreed that there are use cases for
> third-party vendors building tools around Kafka. We haven't reached the
> conclusion whether the added complexity justifies the use cases. We will
> follow up on the mailing list with use cases, container format people
> have
> been using, and details on the proposal.
>
>
> The video will be uploaded soon in https://cwiki.apache.org/
> confluence/display/KAFKA/Kafka+Improvement+Proposals .
>
> Thanks,
>
> Jun
>
> On Mon, Oct 17, 2016 at 10:49 AM, Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Everyone.,
> >
> > We plan to have a Kafka KIP meeting this coming Wednesday at 11:00am PST.
> > If you plan to attend but haven't received an invite, please let me know.
> > The following is the tentative agenda.
> >
> > Agenda:
> > KIP-82: add record header
> >
> > Thanks,
> >
> > Jun
> >
> The information contained in this email is strictly confidential and for
> the use of the addressee only, unless otherwise indicated. If you are not
> the intended recipient, please do not read, copy, use or disclose to others
> this message or any attachment. Please also notify the sender by replying
> to this email or by telephone (+44(020 7896 0011) and then delete the email
> and any copies of it. Opinions, conclusion (etc) that do not relate to the
> official business of this company shall be understood as neither given nor
> endorsed by it. IG is a trading name of IG Markets Limited (a company
> registered in England and Wales, company number 04008957) and IG Index
> Limited (a company registered in England and Wales, company number
> 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
> London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG
> Index Limited (register number 114059) are authorised and regulated by the
> Financial Conduct Authority.
>
Re: Kafka KIP meeting Oct 19 at 11:00am PST
Posted by Jim Jagielski <ji...@jaguNET.com>.
++1
> On Oct 21, 2016, at 11:47 AM, Gwen Shapira <gw...@confluent.io> wrote:
>
> I think this will be a good idea. It will separate an issue with concrete
> use-case and a major pain-point from a wider discussion about architectures
> and who is responsible for data formats.
>
>
> On Fri, Oct 21, 2016 at 12:57 AM, Michael Pearce <Mi...@ig.com>
> wrote:
>
>> I had noted that what ever the solution having compaction based on null
>> payload was agreed isn't elegant.
>>
>> Shall we raise another kip to : as discussed propose using a attribute bit
>> for delete/compaction flag as well/or instead of null value and updating
>> compaction logic to look at that delelete/compaction attribute
>>
>> I believe this is less contentious, so that at least we get that done
>> alleviating some concerns whilst the below gets discussed further?
>>
>> ________________________________________
>> From: Jun Rao <ju...@confluent.io>
>> Sent: Wednesday, October 19, 2016 8:56:52 PM
>> To: dev@kafka.apache.org
>> Subject: Re: Kafka KIP meeting Oct 19 at 11:00am PST
>>
>> The following are the notes from today's KIP discussion.
>>
>>
>> - KIP-82 - add record header: We agreed that there are use cases for
>> third-party vendors building tools around Kafka. We haven't reached the
>> conclusion whether the added complexity justifies the use cases. We will
>> follow up on the mailing list with use cases, container format people
>> have
>> been using, and details on the proposal.
>>
>>
>> The video will be uploaded soon in https://cwiki.apache.org/
>> confluence/display/KAFKA/Kafka+Improvement+Proposals .
>>
>> Thanks,
>>
>> Jun
>>
>> On Mon, Oct 17, 2016 at 10:49 AM, Jun Rao <ju...@confluent.io> wrote:
>>
>>> Hi, Everyone.,
>>>
>>> We plan to have a Kafka KIP meeting this coming Wednesday at 11:00am PST.
>>> If you plan to attend but haven't received an invite, please let me know.
>>> The following is the tentative agenda.
>>>
>>> Agenda:
>>> KIP-82: add record header
>>>
>>> Thanks,
>>>
>>> Jun
>>>
>> The information contained in this email is strictly confidential and for
>> the use of the addressee only, unless otherwise indicated. If you are not
>> the intended recipient, please do not read, copy, use or disclose to others
>> this message or any attachment. Please also notify the sender by replying
>> to this email or by telephone (+44(020 7896 0011) and then delete the email
>> and any copies of it. Opinions, conclusion (etc) that do not relate to the
>> official business of this company shall be understood as neither given nor
>> endorsed by it. IG is a trading name of IG Markets Limited (a company
>> registered in England and Wales, company number 04008957) and IG Index
>> Limited (a company registered in England and Wales, company number
>> 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
>> London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG
>> Index Limited (register number 114059) are authorised and regulated by the
>> Financial Conduct Authority.
>>
>
>
>
> --
> *Gwen Shapira*
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> <http://www.confluent.io/blog>
Re: Kafka KIP meeting Oct 19 at 11:00am PST
Posted by Gwen Shapira <gw...@confluent.io>.
I think this will be a good idea. It will separate an issue with concrete
use-case and a major pain-point from a wider discussion about architectures
and who is responsible for data formats.
On Fri, Oct 21, 2016 at 12:57 AM, Michael Pearce <Mi...@ig.com>
wrote:
> I had noted that what ever the solution having compaction based on null
> payload was agreed isn't elegant.
>
> Shall we raise another kip to : as discussed propose using a attribute bit
> for delete/compaction flag as well/or instead of null value and updating
> compaction logic to look at that delelete/compaction attribute
>
> I believe this is less contentious, so that at least we get that done
> alleviating some concerns whilst the below gets discussed further?
>
> ________________________________________
> From: Jun Rao <ju...@confluent.io>
> Sent: Wednesday, October 19, 2016 8:56:52 PM
> To: dev@kafka.apache.org
> Subject: Re: Kafka KIP meeting Oct 19 at 11:00am PST
>
> The following are the notes from today's KIP discussion.
>
>
> - KIP-82 - add record header: We agreed that there are use cases for
> third-party vendors building tools around Kafka. We haven't reached the
> conclusion whether the added complexity justifies the use cases. We will
> follow up on the mailing list with use cases, container format people
> have
> been using, and details on the proposal.
>
>
> The video will be uploaded soon in https://cwiki.apache.org/
> confluence/display/KAFKA/Kafka+Improvement+Proposals .
>
> Thanks,
>
> Jun
>
> On Mon, Oct 17, 2016 at 10:49 AM, Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Everyone.,
> >
> > We plan to have a Kafka KIP meeting this coming Wednesday at 11:00am PST.
> > If you plan to attend but haven't received an invite, please let me know.
> > The following is the tentative agenda.
> >
> > Agenda:
> > KIP-82: add record header
> >
> > Thanks,
> >
> > Jun
> >
> The information contained in this email is strictly confidential and for
> the use of the addressee only, unless otherwise indicated. If you are not
> the intended recipient, please do not read, copy, use or disclose to others
> this message or any attachment. Please also notify the sender by replying
> to this email or by telephone (+44(020 7896 0011) and then delete the email
> and any copies of it. Opinions, conclusion (etc) that do not relate to the
> official business of this company shall be understood as neither given nor
> endorsed by it. IG is a trading name of IG Markets Limited (a company
> registered in England and Wales, company number 04008957) and IG Index
> Limited (a company registered in England and Wales, company number
> 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
> London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG
> Index Limited (register number 114059) are authorised and regulated by the
> Financial Conduct Authority.
>
--
*Gwen Shapira*
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
<http://www.confluent.io/blog>
Re: Kafka KIP meeting Oct 19 at 11:00am PST
Posted by Michael Pearce <Mi...@ig.com>.
I had noted that what ever the solution having compaction based on null payload was agreed isn't elegant.
Shall we raise another kip to : as discussed propose using a attribute bit for delete/compaction flag as well/or instead of null value and updating compaction logic to look at that delelete/compaction attribute
I believe this is less contentious, so that at least we get that done alleviating some concerns whilst the below gets discussed further?
________________________________________
From: Jun Rao <ju...@confluent.io>
Sent: Wednesday, October 19, 2016 8:56:52 PM
To: dev@kafka.apache.org
Subject: Re: Kafka KIP meeting Oct 19 at 11:00am PST
The following are the notes from today's KIP discussion.
- KIP-82 - add record header: We agreed that there are use cases for
third-party vendors building tools around Kafka. We haven't reached the
conclusion whether the added complexity justifies the use cases. We will
follow up on the mailing list with use cases, container format people have
been using, and details on the proposal.
The video will be uploaded soon in https://cwiki.apache.org/
confluence/display/KAFKA/Kafka+Improvement+Proposals .
Thanks,
Jun
On Mon, Oct 17, 2016 at 10:49 AM, Jun Rao <ju...@confluent.io> wrote:
> Hi, Everyone.,
>
> We plan to have a Kafka KIP meeting this coming Wednesday at 11:00am PST.
> If you plan to attend but haven't received an invite, please let me know.
> The following is the tentative agenda.
>
> Agenda:
> KIP-82: add record header
>
> Thanks,
>
> Jun
>
The information contained in this email is strictly confidential and for the use of the addressee only, unless otherwise indicated. If you are not the intended recipient, please do not read, copy, use or disclose to others this message or any attachment. Please also notify the sender by replying to this email or by telephone (+44(020 7896 0011) and then delete the email and any copies of it. Opinions, conclusion (etc) that do not relate to the official business of this company shall be understood as neither given nor endorsed by it. IG is a trading name of IG Markets Limited (a company registered in England and Wales, company number 04008957) and IG Index Limited (a company registered in England and Wales, company number 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill, London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG Index Limited (register number 114059) are authorised and regulated by the Financial Conduct Authority.
Re: Kafka KIP meeting Oct 19 at 11:00am PST
Posted by Jun Rao <ju...@confluent.io>.
The following are the notes from today's KIP discussion.
- KIP-82 - add record header: We agreed that there are use cases for
third-party vendors building tools around Kafka. We haven't reached the
conclusion whether the added complexity justifies the use cases. We will
follow up on the mailing list with use cases, container format people have
been using, and details on the proposal.
The video will be uploaded soon in https://cwiki.apache.org/
confluence/display/KAFKA/Kafka+Improvement+Proposals .
Thanks,
Jun
On Mon, Oct 17, 2016 at 10:49 AM, Jun Rao <ju...@confluent.io> wrote:
> Hi, Everyone.,
>
> We plan to have a Kafka KIP meeting this coming Wednesday at 11:00am PST.
> If you plan to attend but haven't received an invite, please let me know.
> The following is the tentative agenda.
>
> Agenda:
> KIP-82: add record header
>
> Thanks,
>
> Jun
>