You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Arjun Satish <ar...@gmail.com> on 2018/05/17 07:58:04 UTC
[VOTE] KIP-298: Error Handling in Connect kafka
All,
Many thanks for all the feedback on KIP-298. Highly appreciate the time and
effort you all put into it.
I've updated the KIP accordingly, and would like to start to start a vote
on it.
KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-
298%3A+Error+Handling+in+Connect
JIRA: https://issues.apache.org/jira/browse/KAFKA-6738
Discussion Thread: https://www.mail-archive.com/
dev@kafka.apache.org/msg87660.html
Thanks very much!
Re: [VOTE] KIP-298: Error Handling in Connect kafka
Posted by Arjun Satish <ar...@gmail.com>.
All,
The PR discussions prompted us to consider adding a new configuration
property for the dead letter queue. This (boolean) property when enabled,
would add headers to the raw record that contain details about the error
(such as original topic, partition, offset of the record, connector name,
task number, processing stage, exception class, and error message). By
default, the property would be set to false and no headers would be added.
I understand that the KIP freeze deadline has passed, but this minor
feature brings good value to the dead letter queue.
Please find the updates to the KIP in the section here, and let me know
what you think:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-
298%3A+Error+Handling+in+Connect#KIP-298:ErrorHandlinginConnect-
DeadLetterQueue(forSinkConnectorsonly)
Thanks very much,
On Tue, May 22, 2018 at 2:37 PM, Arjun Satish <ar...@gmail.com>
wrote:
> All,
>
> I'm closing this voting thread with 5 binding votes (Gwen, Guozhang,
> Matthias, Ewen, Jason) and 3 non-binding votes (Konstantine, Magesh,
> Randall).
>
> Thanks everyone for your comments and feedback. Much appreciated!
>
> Best,
> Arjun
>
> On Tue, May 22, 2018 at 11:39 AM, Jason Gustafson <ja...@confluent.io>
> wrote:
>
>> +1. Thanks for the KIP!
>>
>> On Mon, May 21, 2018 at 1:34 PM, Arjun Satish <ar...@gmail.com>
>> wrote:
>>
>> > All,
>> >
>> > Thanks so much for your feedback on this KIP. I've made some small
>> > modifications today. I'll wait till midnight today (PDT) to close this
>> > vote. Please let me know if there are any further comments.
>> >
>> > Best,
>> >
>> > On Mon, May 21, 2018 at 11:29 AM, Ewen Cheslack-Postava <
>> ewen@confluent.io
>> > >
>> > wrote:
>> >
>> > > +1 binding. I had one last comment in the DISCUSS thread, but not
>> really
>> > a
>> > > blocker.
>> > >
>> > > -Ewen
>> > >
>> > > On Mon, May 21, 2018 at 9:48 AM Matthias J. Sax <
>> matthias@confluent.io>
>> > > wrote:
>> > >
>> > > > +1 (binding)
>> > > >
>> > > >
>> > > >
>> > > > On 5/21/18 9:30 AM, Randall Hauch wrote:
>> > > > > Thanks, Arjun. +1 (non-binding)
>> > > > >
>> > > > > Regards,
>> > > > > Randall
>> > > > >
>> > > > > On Mon, May 21, 2018 at 11:14 AM, Guozhang Wang <
>> wangguoz@gmail.com>
>> > > > wrote:
>> > > > >
>> > > > >> Thanks for the KIP. +1 (binding)
>> > > > >>
>> > > > >>
>> > > > >> Guozhang
>> > > > >>
>> > > > >> On Fri, May 18, 2018 at 3:36 PM, Gwen Shapira <gwen@confluent.io
>> >
>> > > > wrote:
>> > > > >>
>> > > > >>> +1
>> > > > >>>
>> > > > >>> Thank you! Error handling in Connect will be a huge improvement.
>> > > > >>>
>> > > > >>> On Thu, May 17, 2018, 1:58 AM Arjun Satish <
>> arjun.satish@gmail.com
>> > >
>> > > > >> wrote:
>> > > > >>>
>> > > > >>>> All,
>> > > > >>>>
>> > > > >>>> Many thanks for all the feedback on KIP-298. Highly appreciate
>> the
>> > > > time
>> > > > >>> and
>> > > > >>>> effort you all put into it.
>> > > > >>>>
>> > > > >>>> I've updated the KIP accordingly, and would like to start to
>> > start a
>> > > > >> vote
>> > > > >>>> on it.
>> > > > >>>>
>> > > > >>>> KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> > > > >>>> 298%3A+Error+Handling+in+Connect
>> > > > >>>> JIRA: https://issues.apache.org/jira/browse/KAFKA-6738
>> > > > >>>> Discussion Thread: https://www.mail-archive.com/
>> > > > >>>> dev@kafka.apache.org/msg87660.html
>> > > > >>>>
>> > > > >>>> Thanks very much!
>> > > > >>>>
>> > > > >>>
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >> --
>> > > > >> -- Guozhang
>> > > > >>
>> > > > >
>> > > >
>> > > >
>> > >
>> >
>>
>
>
Re: [VOTE] KIP-298: Error Handling in Connect kafka
Posted by Arjun Satish <ar...@gmail.com>.
All,
I'm closing this voting thread with 5 binding votes (Gwen, Guozhang,
Matthias, Ewen, Jason) and 3 non-binding votes (Konstantine, Magesh,
Randall).
Thanks everyone for your comments and feedback. Much appreciated!
Best,
Arjun
On Tue, May 22, 2018 at 11:39 AM, Jason Gustafson <ja...@confluent.io>
wrote:
> +1. Thanks for the KIP!
>
> On Mon, May 21, 2018 at 1:34 PM, Arjun Satish <ar...@gmail.com>
> wrote:
>
> > All,
> >
> > Thanks so much for your feedback on this KIP. I've made some small
> > modifications today. I'll wait till midnight today (PDT) to close this
> > vote. Please let me know if there are any further comments.
> >
> > Best,
> >
> > On Mon, May 21, 2018 at 11:29 AM, Ewen Cheslack-Postava <
> ewen@confluent.io
> > >
> > wrote:
> >
> > > +1 binding. I had one last comment in the DISCUSS thread, but not
> really
> > a
> > > blocker.
> > >
> > > -Ewen
> > >
> > > On Mon, May 21, 2018 at 9:48 AM Matthias J. Sax <matthias@confluent.io
> >
> > > wrote:
> > >
> > > > +1 (binding)
> > > >
> > > >
> > > >
> > > > On 5/21/18 9:30 AM, Randall Hauch wrote:
> > > > > Thanks, Arjun. +1 (non-binding)
> > > > >
> > > > > Regards,
> > > > > Randall
> > > > >
> > > > > On Mon, May 21, 2018 at 11:14 AM, Guozhang Wang <
> wangguoz@gmail.com>
> > > > wrote:
> > > > >
> > > > >> Thanks for the KIP. +1 (binding)
> > > > >>
> > > > >>
> > > > >> Guozhang
> > > > >>
> > > > >> On Fri, May 18, 2018 at 3:36 PM, Gwen Shapira <gw...@confluent.io>
> > > > wrote:
> > > > >>
> > > > >>> +1
> > > > >>>
> > > > >>> Thank you! Error handling in Connect will be a huge improvement.
> > > > >>>
> > > > >>> On Thu, May 17, 2018, 1:58 AM Arjun Satish <
> arjun.satish@gmail.com
> > >
> > > > >> wrote:
> > > > >>>
> > > > >>>> All,
> > > > >>>>
> > > > >>>> Many thanks for all the feedback on KIP-298. Highly appreciate
> the
> > > > time
> > > > >>> and
> > > > >>>> effort you all put into it.
> > > > >>>>
> > > > >>>> I've updated the KIP accordingly, and would like to start to
> > start a
> > > > >> vote
> > > > >>>> on it.
> > > > >>>>
> > > > >>>> KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > >>>> 298%3A+Error+Handling+in+Connect
> > > > >>>> JIRA: https://issues.apache.org/jira/browse/KAFKA-6738
> > > > >>>> Discussion Thread: https://www.mail-archive.com/
> > > > >>>> dev@kafka.apache.org/msg87660.html
> > > > >>>>
> > > > >>>> Thanks very much!
> > > > >>>>
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> -- Guozhang
> > > > >>
> > > > >
> > > >
> > > >
> > >
> >
>
Re: [VOTE] KIP-298: Error Handling in Connect kafka
Posted by Jason Gustafson <ja...@confluent.io>.
+1. Thanks for the KIP!
On Mon, May 21, 2018 at 1:34 PM, Arjun Satish <ar...@gmail.com>
wrote:
> All,
>
> Thanks so much for your feedback on this KIP. I've made some small
> modifications today. I'll wait till midnight today (PDT) to close this
> vote. Please let me know if there are any further comments.
>
> Best,
>
> On Mon, May 21, 2018 at 11:29 AM, Ewen Cheslack-Postava <ewen@confluent.io
> >
> wrote:
>
> > +1 binding. I had one last comment in the DISCUSS thread, but not really
> a
> > blocker.
> >
> > -Ewen
> >
> > On Mon, May 21, 2018 at 9:48 AM Matthias J. Sax <ma...@confluent.io>
> > wrote:
> >
> > > +1 (binding)
> > >
> > >
> > >
> > > On 5/21/18 9:30 AM, Randall Hauch wrote:
> > > > Thanks, Arjun. +1 (non-binding)
> > > >
> > > > Regards,
> > > > Randall
> > > >
> > > > On Mon, May 21, 2018 at 11:14 AM, Guozhang Wang <wa...@gmail.com>
> > > wrote:
> > > >
> > > >> Thanks for the KIP. +1 (binding)
> > > >>
> > > >>
> > > >> Guozhang
> > > >>
> > > >> On Fri, May 18, 2018 at 3:36 PM, Gwen Shapira <gw...@confluent.io>
> > > wrote:
> > > >>
> > > >>> +1
> > > >>>
> > > >>> Thank you! Error handling in Connect will be a huge improvement.
> > > >>>
> > > >>> On Thu, May 17, 2018, 1:58 AM Arjun Satish <arjun.satish@gmail.com
> >
> > > >> wrote:
> > > >>>
> > > >>>> All,
> > > >>>>
> > > >>>> Many thanks for all the feedback on KIP-298. Highly appreciate the
> > > time
> > > >>> and
> > > >>>> effort you all put into it.
> > > >>>>
> > > >>>> I've updated the KIP accordingly, and would like to start to
> start a
> > > >> vote
> > > >>>> on it.
> > > >>>>
> > > >>>> KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > >>>> 298%3A+Error+Handling+in+Connect
> > > >>>> JIRA: https://issues.apache.org/jira/browse/KAFKA-6738
> > > >>>> Discussion Thread: https://www.mail-archive.com/
> > > >>>> dev@kafka.apache.org/msg87660.html
> > > >>>>
> > > >>>> Thanks very much!
> > > >>>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> -- Guozhang
> > > >>
> > > >
> > >
> > >
> >
>
Re: [VOTE] KIP-298: Error Handling in Connect kafka
Posted by Arjun Satish <ar...@gmail.com>.
All,
Thanks so much for your feedback on this KIP. I've made some small
modifications today. I'll wait till midnight today (PDT) to close this
vote. Please let me know if there are any further comments.
Best,
On Mon, May 21, 2018 at 11:29 AM, Ewen Cheslack-Postava <ew...@confluent.io>
wrote:
> +1 binding. I had one last comment in the DISCUSS thread, but not really a
> blocker.
>
> -Ewen
>
> On Mon, May 21, 2018 at 9:48 AM Matthias J. Sax <ma...@confluent.io>
> wrote:
>
> > +1 (binding)
> >
> >
> >
> > On 5/21/18 9:30 AM, Randall Hauch wrote:
> > > Thanks, Arjun. +1 (non-binding)
> > >
> > > Regards,
> > > Randall
> > >
> > > On Mon, May 21, 2018 at 11:14 AM, Guozhang Wang <wa...@gmail.com>
> > wrote:
> > >
> > >> Thanks for the KIP. +1 (binding)
> > >>
> > >>
> > >> Guozhang
> > >>
> > >> On Fri, May 18, 2018 at 3:36 PM, Gwen Shapira <gw...@confluent.io>
> > wrote:
> > >>
> > >>> +1
> > >>>
> > >>> Thank you! Error handling in Connect will be a huge improvement.
> > >>>
> > >>> On Thu, May 17, 2018, 1:58 AM Arjun Satish <ar...@gmail.com>
> > >> wrote:
> > >>>
> > >>>> All,
> > >>>>
> > >>>> Many thanks for all the feedback on KIP-298. Highly appreciate the
> > time
> > >>> and
> > >>>> effort you all put into it.
> > >>>>
> > >>>> I've updated the KIP accordingly, and would like to start to start a
> > >> vote
> > >>>> on it.
> > >>>>
> > >>>> KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > >>>> 298%3A+Error+Handling+in+Connect
> > >>>> JIRA: https://issues.apache.org/jira/browse/KAFKA-6738
> > >>>> Discussion Thread: https://www.mail-archive.com/
> > >>>> dev@kafka.apache.org/msg87660.html
> > >>>>
> > >>>> Thanks very much!
> > >>>>
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> -- Guozhang
> > >>
> > >
> >
> >
>
Re: [VOTE] KIP-298: Error Handling in Connect kafka
Posted by Ewen Cheslack-Postava <ew...@confluent.io>.
+1 binding. I had one last comment in the DISCUSS thread, but not really a
blocker.
-Ewen
On Mon, May 21, 2018 at 9:48 AM Matthias J. Sax <ma...@confluent.io>
wrote:
> +1 (binding)
>
>
>
> On 5/21/18 9:30 AM, Randall Hauch wrote:
> > Thanks, Arjun. +1 (non-binding)
> >
> > Regards,
> > Randall
> >
> > On Mon, May 21, 2018 at 11:14 AM, Guozhang Wang <wa...@gmail.com>
> wrote:
> >
> >> Thanks for the KIP. +1 (binding)
> >>
> >>
> >> Guozhang
> >>
> >> On Fri, May 18, 2018 at 3:36 PM, Gwen Shapira <gw...@confluent.io>
> wrote:
> >>
> >>> +1
> >>>
> >>> Thank you! Error handling in Connect will be a huge improvement.
> >>>
> >>> On Thu, May 17, 2018, 1:58 AM Arjun Satish <ar...@gmail.com>
> >> wrote:
> >>>
> >>>> All,
> >>>>
> >>>> Many thanks for all the feedback on KIP-298. Highly appreciate the
> time
> >>> and
> >>>> effort you all put into it.
> >>>>
> >>>> I've updated the KIP accordingly, and would like to start to start a
> >> vote
> >>>> on it.
> >>>>
> >>>> KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>>> 298%3A+Error+Handling+in+Connect
> >>>> JIRA: https://issues.apache.org/jira/browse/KAFKA-6738
> >>>> Discussion Thread: https://www.mail-archive.com/
> >>>> dev@kafka.apache.org/msg87660.html
> >>>>
> >>>> Thanks very much!
> >>>>
> >>>
> >>
> >>
> >>
> >> --
> >> -- Guozhang
> >>
> >
>
>
Re: [VOTE] KIP-298: Error Handling in Connect kafka
Posted by "Matthias J. Sax" <ma...@confluent.io>.
+1 (binding)
On 5/21/18 9:30 AM, Randall Hauch wrote:
> Thanks, Arjun. +1 (non-binding)
>
> Regards,
> Randall
>
> On Mon, May 21, 2018 at 11:14 AM, Guozhang Wang <wa...@gmail.com> wrote:
>
>> Thanks for the KIP. +1 (binding)
>>
>>
>> Guozhang
>>
>> On Fri, May 18, 2018 at 3:36 PM, Gwen Shapira <gw...@confluent.io> wrote:
>>
>>> +1
>>>
>>> Thank you! Error handling in Connect will be a huge improvement.
>>>
>>> On Thu, May 17, 2018, 1:58 AM Arjun Satish <ar...@gmail.com>
>> wrote:
>>>
>>>> All,
>>>>
>>>> Many thanks for all the feedback on KIP-298. Highly appreciate the time
>>> and
>>>> effort you all put into it.
>>>>
>>>> I've updated the KIP accordingly, and would like to start to start a
>> vote
>>>> on it.
>>>>
>>>> KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>>> 298%3A+Error+Handling+in+Connect
>>>> JIRA: https://issues.apache.org/jira/browse/KAFKA-6738
>>>> Discussion Thread: https://www.mail-archive.com/
>>>> dev@kafka.apache.org/msg87660.html
>>>>
>>>> Thanks very much!
>>>>
>>>
>>
>>
>>
>> --
>> -- Guozhang
>>
>
Re: [VOTE] KIP-298: Error Handling in Connect kafka
Posted by Randall Hauch <rh...@gmail.com>.
Thanks, Arjun. +1 (non-binding)
Regards,
Randall
On Mon, May 21, 2018 at 11:14 AM, Guozhang Wang <wa...@gmail.com> wrote:
> Thanks for the KIP. +1 (binding)
>
>
> Guozhang
>
> On Fri, May 18, 2018 at 3:36 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > +1
> >
> > Thank you! Error handling in Connect will be a huge improvement.
> >
> > On Thu, May 17, 2018, 1:58 AM Arjun Satish <ar...@gmail.com>
> wrote:
> >
> > > All,
> > >
> > > Many thanks for all the feedback on KIP-298. Highly appreciate the time
> > and
> > > effort you all put into it.
> > >
> > > I've updated the KIP accordingly, and would like to start to start a
> vote
> > > on it.
> > >
> > > KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 298%3A+Error+Handling+in+Connect
> > > JIRA: https://issues.apache.org/jira/browse/KAFKA-6738
> > > Discussion Thread: https://www.mail-archive.com/
> > > dev@kafka.apache.org/msg87660.html
> > >
> > > Thanks very much!
> > >
> >
>
>
>
> --
> -- Guozhang
>
Re: [VOTE] KIP-298: Error Handling in Connect kafka
Posted by Guozhang Wang <wa...@gmail.com>.
Thanks for the KIP. +1 (binding)
Guozhang
On Fri, May 18, 2018 at 3:36 PM, Gwen Shapira <gw...@confluent.io> wrote:
> +1
>
> Thank you! Error handling in Connect will be a huge improvement.
>
> On Thu, May 17, 2018, 1:58 AM Arjun Satish <ar...@gmail.com> wrote:
>
> > All,
> >
> > Many thanks for all the feedback on KIP-298. Highly appreciate the time
> and
> > effort you all put into it.
> >
> > I've updated the KIP accordingly, and would like to start to start a vote
> > on it.
> >
> > KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 298%3A+Error+Handling+in+Connect
> > JIRA: https://issues.apache.org/jira/browse/KAFKA-6738
> > Discussion Thread: https://www.mail-archive.com/
> > dev@kafka.apache.org/msg87660.html
> >
> > Thanks very much!
> >
>
--
-- Guozhang
Re: [VOTE] KIP-298: Error Handling in Connect kafka
Posted by Gwen Shapira <gw...@confluent.io>.
+1
Thank you! Error handling in Connect will be a huge improvement.
On Thu, May 17, 2018, 1:58 AM Arjun Satish <ar...@gmail.com> wrote:
> All,
>
> Many thanks for all the feedback on KIP-298. Highly appreciate the time and
> effort you all put into it.
>
> I've updated the KIP accordingly, and would like to start to start a vote
> on it.
>
> KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 298%3A+Error+Handling+in+Connect
> JIRA: https://issues.apache.org/jira/browse/KAFKA-6738
> Discussion Thread: https://www.mail-archive.com/
> dev@kafka.apache.org/msg87660.html
>
> Thanks very much!
>
Re: [VOTE] KIP-298: Error Handling in Connect kafka
Posted by Magesh Nandakumar <ma...@confluent.io>.
+1 (non-binding)
Thanks,
Magesh
On Fri, May 18, 2018 at 2:38 PM, Konstantine Karantasis <
konstantine@confluent.io> wrote:
> +1 (non-binding)
>
> - Konstantine
>
> On Thu, May 17, 2018 at 12:58 AM, Arjun Satish <ar...@gmail.com>
> wrote:
>
> > All,
> >
> > Many thanks for all the feedback on KIP-298. Highly appreciate the time
> and
> > effort you all put into it.
> >
> > I've updated the KIP accordingly, and would like to start to start a vote
> > on it.
> >
> > KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 298%3A+Error+Handling+in+Connect
> > JIRA: https://issues.apache.org/jira/browse/KAFKA-6738
> > Discussion Thread: https://www.mail-archive.com/
> > dev@kafka.apache.org/msg87660.html
> >
> > Thanks very much!
> >
>
Re: [VOTE] KIP-298: Error Handling in Connect kafka
Posted by Konstantine Karantasis <ko...@confluent.io>.
+1 (non-binding)
- Konstantine
On Thu, May 17, 2018 at 12:58 AM, Arjun Satish <ar...@gmail.com>
wrote:
> All,
>
> Many thanks for all the feedback on KIP-298. Highly appreciate the time and
> effort you all put into it.
>
> I've updated the KIP accordingly, and would like to start to start a vote
> on it.
>
> KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 298%3A+Error+Handling+in+Connect
> JIRA: https://issues.apache.org/jira/browse/KAFKA-6738
> Discussion Thread: https://www.mail-archive.com/
> dev@kafka.apache.org/msg87660.html
>
> Thanks very much!
>