You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by huxi_2b <hu...@hotmail.com> on 2020/07/10 04:57:28 UTC
Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord
fields in DefaultMessageFormatter
Badai,
Replying in this thread. Two possible ways come up into my head:
1. CreateTime:1592475472398|key1|3|<0>|h1=v1,h2=v2|value1
2. CreateTime:1592475472398|key1|3|offset=0|h1=v1,h2=v2|value1
I prefer the option #1 which could be accomplished by a complementary usage description. What do you think?
On 2020/06/21 13:39:36, Badai Aqrandista <b....@confluent.io> wrote:
> Excellent.>
>
> Would like to hear more feedback from others.>
>
> On Sat, Jun 20, 2020 at 1:27 AM David Jacot <dj...@confluent.io> wrote:>
> >>
> > Hi Badai,>
> >>
> > Thanks for your reply.>
> >>
> > 2. Yes, that makes sense.>
> >>
> > Best,>
> > David>
> >>
> > On Thu, Jun 18, 2020 at 2:08 PM Badai Aqrandista <ba...@confluent.io> wrote:>
> >>
> > > David>
> > >>
> > > Thank you for replying>
> > >>
> > > 1. It seems that `print.partition` is already implemented. Do you confirm?>
> > > BADAI: Yes, you are correct. I have removed it from the KIP.>
> > >>
> > > 2. Will `null.literal` be only used when the value of the message>
> > > is NULL or for any fields? Also, it seems that we print out "null">
> > > today when the key or the value is empty. Shall we use "null" as>
> > > a default instead of ""?>
> > > BADAI: For any fields. Do you think this is useful?>
> > >>
> > > 3. Could we add a small example of the output in the KIP?>
> > > BADAI: Yes, I have updated the KIP to add a couple of example.>
> > >>
> > > 4. When there are no headers, are we going to print something>
> > > to indicate it to the user? For instance, we print out NO_TIMESTAMP>
> > > where there is no timestamp.>
> > > BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.>
> > >>
> > > Thanks>
> > > Badai>
> > >>
> > >>
> > > On Thu, Jun 18, 2020 at 7:25 PM David Jacot <dj...@confluent.io> wrote:>
> > > >>
> > > > Hi Badai,>
> > > >>
> > > > Thanks for resuming this. I have few small comments:>
> > > >>
> > > > 1. It seems that `print.partition` is already implemented. Do you>
> > > confirm?>
> > > >>
> > > > 2. Will `null.literal` be only used when the value of the message>
> > > > is NULL or for any fields? Also, it seems that we print out "null">
> > > > today when the key or the value is empty. Shall we use "null" as>
> > > > a default instead of ""?>
> > > >>
> > > > 3. Could we add a small example of the output in the KIP?>
> > > >>
> > > > 4. When there are no headers, are we going to print something>
> > > > to indicate it to the user? For instance, we print out NO_TIMESTAMP>
> > > > where there is no timestamp.>
> > > >>
> > > > Best,>
> > > > David>
> > > >>
> > > > On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista <ba...@confluent.io>>
> > > wrote:>
> > > >>
> > > > > Hi all,>
> > > > >>
> > > > > I have contacted Mateusz separately and he is ok for me to take over>
> > > > > KIP-431:>
> > > > >>
> > > > >>
> > > > >>
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter>
> > > > >>
> > > > > I have updated it a bit. Can anyone give a quick look at it again and>
> > > > > give me some feedback?>
> > > > >>
> > > > > This feature will be very helpful for people supporting Kafka in>
> > > > > operations.>
> > > > >>
> > > > > If it is ready for a vote, please let me know.>
> > > > >>
> > > > > Thanks>
> > > > > Badai>
> > > > >>
> > > > > On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista <ba...@confluent.io>>
> > > > > wrote:>
> > > > > >>
> > > > > > Mateusz>
> > > > > >>
> > > > > > This KIP would be very useful for debugging. But the last discussion>
> > > > > > is in Feb 2019.>
> > > > > >>
> > > > > > Are you ok if I take over this KIP?>
> > > > > >>
> > > > > > -->
> > > > > > Thanks,>
> > > > > > Badai>
> > > > >>
> > > > >>
> > > > >>
> > > > > -->
> > > > > Thanks,>
> > > > > Badai>
> > > > >>
> > >>
> > >>
> > >>
> > > -->
> > > Thanks,>
> > > Badai>
> > >>
>
>
>
> -- >
> Thanks,>
> Badai>
>
Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord
fields in DefaultMessageFormatter
Posted by Badai Aqrandista <ba...@confluent.io>.
Hi all
The vote for KIP-431 has passed with 3 binding and 1 non-binding +1s, and
no objections.
Thanks everyone for reviews and feedback.
Regards,
Badai
On Fri, Jul 10, 2020 at 5:22 PM Badai Aqrandista <ba...@confluent.io> wrote:
>
> Hu
>
> I would make it more readable and move the key next to the value like this:
>
> CreateTime:1592475472398|Partition:3|Offset:0|Headers:h1=v1,h2=v2|key1|value1
>
> What do you think?
>
> In future KIP, we'll add some formatting like David Jacot suggested.
>
> Regards
> Badai
>
> On Fri, Jul 10, 2020 at 2:57 PM huxi_2b <hu...@hotmail.com> wrote:
> >
> > Badai,
> >
> > Replying in this thread. Two possible ways come up into my head:
> >
> > 1. CreateTime:1592475472398|key1|3|<0>|h1=v1,h2=v2|value1
> > 2. CreateTime:1592475472398|key1|3|offset=0|h1=v1,h2=v2|value1
> >
> > I prefer the option #1 which could be accomplished by a complementary usage description. What do you think?
> >
> > On 2020/06/21 13:39:36, Badai Aqrandista <b....@confluent.io> wrote:
> > > Excellent.>
> > >
> > > Would like to hear more feedback from others.>
> > >
> > > On Sat, Jun 20, 2020 at 1:27 AM David Jacot <dj...@confluent.io> wrote:>
> > > >>
> > > > Hi Badai,>
> > > >>
> > > > Thanks for your reply.>
> > > >>
> > > > 2. Yes, that makes sense.>
> > > >>
> > > > Best,>
> > > > David>
> > > >>
> > > > On Thu, Jun 18, 2020 at 2:08 PM Badai Aqrandista <ba...@confluent.io> wrote:>
> > > >>
> > > > > David>
> > > > >>
> > > > > Thank you for replying>
> > > > >>
> > > > > 1. It seems that `print.partition` is already implemented. Do you confirm?>
> > > > > BADAI: Yes, you are correct. I have removed it from the KIP.>
> > > > >>
> > > > > 2. Will `null.literal` be only used when the value of the message>
> > > > > is NULL or for any fields? Also, it seems that we print out "null">
> > > > > today when the key or the value is empty. Shall we use "null" as>
> > > > > a default instead of ""?>
> > > > > BADAI: For any fields. Do you think this is useful?>
> > > > >>
> > > > > 3. Could we add a small example of the output in the KIP?>
> > > > > BADAI: Yes, I have updated the KIP to add a couple of example.>
> > > > >>
> > > > > 4. When there are no headers, are we going to print something>
> > > > > to indicate it to the user? For instance, we print out NO_TIMESTAMP>
> > > > > where there is no timestamp.>
> > > > > BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.>
> > > > >>
> > > > > Thanks>
> > > > > Badai>
> > > > >>
> > > > >>
> > > > > On Thu, Jun 18, 2020 at 7:25 PM David Jacot <dj...@confluent.io> wrote:>
> > > > > >>
> > > > > > Hi Badai,>
> > > > > >>
> > > > > > Thanks for resuming this. I have few small comments:>
> > > > > >>
> > > > > > 1. It seems that `print.partition` is already implemented. Do you>
> > > > > confirm?>
> > > > > >>
> > > > > > 2. Will `null.literal` be only used when the value of the message>
> > > > > > is NULL or for any fields? Also, it seems that we print out "null">
> > > > > > today when the key or the value is empty. Shall we use "null" as>
> > > > > > a default instead of ""?>
> > > > > >>
> > > > > > 3. Could we add a small example of the output in the KIP?>
> > > > > >>
> > > > > > 4. When there are no headers, are we going to print something>
> > > > > > to indicate it to the user? For instance, we print out NO_TIMESTAMP>
> > > > > > where there is no timestamp.>
> > > > > >>
> > > > > > Best,>
> > > > > > David>
> > > > > >>
> > > > > > On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista <ba...@confluent.io>>
> > > > > wrote:>
> > > > > >>
> > > > > > > Hi all,>
> > > > > > >>
> > > > > > > I have contacted Mateusz separately and he is ok for me to take over>
> > > > > > > KIP-431:>
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter>
> > > > > > >>
> > > > > > > I have updated it a bit. Can anyone give a quick look at it again and>
> > > > > > > give me some feedback?>
> > > > > > >>
> > > > > > > This feature will be very helpful for people supporting Kafka in>
> > > > > > > operations.>
> > > > > > >>
> > > > > > > If it is ready for a vote, please let me know.>
> > > > > > >>
> > > > > > > Thanks>
> > > > > > > Badai>
> > > > > > >>
> > > > > > > On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista <ba...@confluent.io>>
> > > > > > > wrote:>
> > > > > > > >>
> > > > > > > > Mateusz>
> > > > > > > >>
> > > > > > > > This KIP would be very useful for debugging. But the last discussion>
> > > > > > > > is in Feb 2019.>
> > > > > > > >>
> > > > > > > > Are you ok if I take over this KIP?>
> > > > > > > >>
> > > > > > > > -->
> > > > > > > > Thanks,>
> > > > > > > > Badai>
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > > -->
> > > > > > > Thanks,>
> > > > > > > Badai>
> > > > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > > -->
> > > > > Thanks,>
> > > > > Badai>
> > > > >>
> > >
> > >
> > >
> > > -- >
> > > Thanks,>
> > > Badai>
> > >
>
>
>
> --
> Thanks,
> Badai
--
Thanks,
Badai
Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord
fields in DefaultMessageFormatter
Posted by Badai Aqrandista <ba...@confluent.io>.
Hu
I would make it more readable and move the key next to the value like this:
CreateTime:1592475472398|Partition:3|Offset:0|Headers:h1=v1,h2=v2|key1|value1
What do you think?
In future KIP, we'll add some formatting like David Jacot suggested.
Regards
Badai
On Fri, Jul 10, 2020 at 2:57 PM huxi_2b <hu...@hotmail.com> wrote:
>
> Badai,
>
> Replying in this thread. Two possible ways come up into my head:
>
> 1. CreateTime:1592475472398|key1|3|<0>|h1=v1,h2=v2|value1
> 2. CreateTime:1592475472398|key1|3|offset=0|h1=v1,h2=v2|value1
>
> I prefer the option #1 which could be accomplished by a complementary usage description. What do you think?
>
> On 2020/06/21 13:39:36, Badai Aqrandista <b....@confluent.io> wrote:
> > Excellent.>
> >
> > Would like to hear more feedback from others.>
> >
> > On Sat, Jun 20, 2020 at 1:27 AM David Jacot <dj...@confluent.io> wrote:>
> > >>
> > > Hi Badai,>
> > >>
> > > Thanks for your reply.>
> > >>
> > > 2. Yes, that makes sense.>
> > >>
> > > Best,>
> > > David>
> > >>
> > > On Thu, Jun 18, 2020 at 2:08 PM Badai Aqrandista <ba...@confluent.io> wrote:>
> > >>
> > > > David>
> > > >>
> > > > Thank you for replying>
> > > >>
> > > > 1. It seems that `print.partition` is already implemented. Do you confirm?>
> > > > BADAI: Yes, you are correct. I have removed it from the KIP.>
> > > >>
> > > > 2. Will `null.literal` be only used when the value of the message>
> > > > is NULL or for any fields? Also, it seems that we print out "null">
> > > > today when the key or the value is empty. Shall we use "null" as>
> > > > a default instead of ""?>
> > > > BADAI: For any fields. Do you think this is useful?>
> > > >>
> > > > 3. Could we add a small example of the output in the KIP?>
> > > > BADAI: Yes, I have updated the KIP to add a couple of example.>
> > > >>
> > > > 4. When there are no headers, are we going to print something>
> > > > to indicate it to the user? For instance, we print out NO_TIMESTAMP>
> > > > where there is no timestamp.>
> > > > BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.>
> > > >>
> > > > Thanks>
> > > > Badai>
> > > >>
> > > >>
> > > > On Thu, Jun 18, 2020 at 7:25 PM David Jacot <dj...@confluent.io> wrote:>
> > > > >>
> > > > > Hi Badai,>
> > > > >>
> > > > > Thanks for resuming this. I have few small comments:>
> > > > >>
> > > > > 1. It seems that `print.partition` is already implemented. Do you>
> > > > confirm?>
> > > > >>
> > > > > 2. Will `null.literal` be only used when the value of the message>
> > > > > is NULL or for any fields? Also, it seems that we print out "null">
> > > > > today when the key or the value is empty. Shall we use "null" as>
> > > > > a default instead of ""?>
> > > > >>
> > > > > 3. Could we add a small example of the output in the KIP?>
> > > > >>
> > > > > 4. When there are no headers, are we going to print something>
> > > > > to indicate it to the user? For instance, we print out NO_TIMESTAMP>
> > > > > where there is no timestamp.>
> > > > >>
> > > > > Best,>
> > > > > David>
> > > > >>
> > > > > On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista <ba...@confluent.io>>
> > > > wrote:>
> > > > >>
> > > > > > Hi all,>
> > > > > >>
> > > > > > I have contacted Mateusz separately and he is ok for me to take over>
> > > > > > KIP-431:>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter>
> > > > > >>
> > > > > > I have updated it a bit. Can anyone give a quick look at it again and>
> > > > > > give me some feedback?>
> > > > > >>
> > > > > > This feature will be very helpful for people supporting Kafka in>
> > > > > > operations.>
> > > > > >>
> > > > > > If it is ready for a vote, please let me know.>
> > > > > >>
> > > > > > Thanks>
> > > > > > Badai>
> > > > > >>
> > > > > > On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista <ba...@confluent.io>>
> > > > > > wrote:>
> > > > > > >>
> > > > > > > Mateusz>
> > > > > > >>
> > > > > > > This KIP would be very useful for debugging. But the last discussion>
> > > > > > > is in Feb 2019.>
> > > > > > >>
> > > > > > > Are you ok if I take over this KIP?>
> > > > > > >>
> > > > > > > -->
> > > > > > > Thanks,>
> > > > > > > Badai>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > > -->
> > > > > > Thanks,>
> > > > > > Badai>
> > > > > >>
> > > >>
> > > >>
> > > >>
> > > > -->
> > > > Thanks,>
> > > > Badai>
> > > >>
> >
> >
> >
> > -- >
> > Thanks,>
> > Badai>
> >
--
Thanks,
Badai
Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord
fields in DefaultMessageFormatter
Posted by Badai Aqrandista <ba...@confluent.io>.
Hi all
Does anyone have time to review this PR?
I have created a PR for KIP-431 against the latest trunk:
https://github.com/apache/kafka/pull/9099
Regards
Badai
On Fri, Jul 10, 2020 at 2:57 PM huxi_2b <hu...@hotmail.com> wrote:
>
> Badai,
>
> Replying in this thread. Two possible ways come up into my head:
>
> 1. CreateTime:1592475472398|key1|3|<0>|h1=v1,h2=v2|value1
> 2. CreateTime:1592475472398|key1|3|offset=0|h1=v1,h2=v2|value1
>
> I prefer the option #1 which could be accomplished by a complementary usage description. What do you think?
>
> On 2020/06/21 13:39:36, Badai Aqrandista <b....@confluent.io> wrote:
> > Excellent.>
> >
> > Would like to hear more feedback from others.>
> >
> > On Sat, Jun 20, 2020 at 1:27 AM David Jacot <dj...@confluent.io> wrote:>
> > >>
> > > Hi Badai,>
> > >>
> > > Thanks for your reply.>
> > >>
> > > 2. Yes, that makes sense.>
> > >>
> > > Best,>
> > > David>
> > >>
> > > On Thu, Jun 18, 2020 at 2:08 PM Badai Aqrandista <ba...@confluent.io> wrote:>
> > >>
> > > > David>
> > > >>
> > > > Thank you for replying>
> > > >>
> > > > 1. It seems that `print.partition` is already implemented. Do you confirm?>
> > > > BADAI: Yes, you are correct. I have removed it from the KIP.>
> > > >>
> > > > 2. Will `null.literal` be only used when the value of the message>
> > > > is NULL or for any fields? Also, it seems that we print out "null">
> > > > today when the key or the value is empty. Shall we use "null" as>
> > > > a default instead of ""?>
> > > > BADAI: For any fields. Do you think this is useful?>
> > > >>
> > > > 3. Could we add a small example of the output in the KIP?>
> > > > BADAI: Yes, I have updated the KIP to add a couple of example.>
> > > >>
> > > > 4. When there are no headers, are we going to print something>
> > > > to indicate it to the user? For instance, we print out NO_TIMESTAMP>
> > > > where there is no timestamp.>
> > > > BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.>
> > > >>
> > > > Thanks>
> > > > Badai>
> > > >>
> > > >>
> > > > On Thu, Jun 18, 2020 at 7:25 PM David Jacot <dj...@confluent.io> wrote:>
> > > > >>
> > > > > Hi Badai,>
> > > > >>
> > > > > Thanks for resuming this. I have few small comments:>
> > > > >>
> > > > > 1. It seems that `print.partition` is already implemented. Do you>
> > > > confirm?>
> > > > >>
> > > > > 2. Will `null.literal` be only used when the value of the message>
> > > > > is NULL or for any fields? Also, it seems that we print out "null">
> > > > > today when the key or the value is empty. Shall we use "null" as>
> > > > > a default instead of ""?>
> > > > >>
> > > > > 3. Could we add a small example of the output in the KIP?>
> > > > >>
> > > > > 4. When there are no headers, are we going to print something>
> > > > > to indicate it to the user? For instance, we print out NO_TIMESTAMP>
> > > > > where there is no timestamp.>
> > > > >>
> > > > > Best,>
> > > > > David>
> > > > >>
> > > > > On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista <ba...@confluent.io>>
> > > > wrote:>
> > > > >>
> > > > > > Hi all,>
> > > > > >>
> > > > > > I have contacted Mateusz separately and he is ok for me to take over>
> > > > > > KIP-431:>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter>
> > > > > >>
> > > > > > I have updated it a bit. Can anyone give a quick look at it again and>
> > > > > > give me some feedback?>
> > > > > >>
> > > > > > This feature will be very helpful for people supporting Kafka in>
> > > > > > operations.>
> > > > > >>
> > > > > > If it is ready for a vote, please let me know.>
> > > > > >>
> > > > > > Thanks>
> > > > > > Badai>
> > > > > >>
> > > > > > On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista <ba...@confluent.io>>
> > > > > > wrote:>
> > > > > > >>
> > > > > > > Mateusz>
> > > > > > >>
> > > > > > > This KIP would be very useful for debugging. But the last discussion>
> > > > > > > is in Feb 2019.>
> > > > > > >>
> > > > > > > Are you ok if I take over this KIP?>
> > > > > > >>
> > > > > > > -->
> > > > > > > Thanks,>
> > > > > > > Badai>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > > -->
> > > > > > Thanks,>
> > > > > > Badai>
> > > > > >>
> > > >>
> > > >>
> > > >>
> > > > -->
> > > > Thanks,>
> > > > Badai>
> > > >>
> >
> >
> >
> > -- >
> > Thanks,>
> > Badai>
> >
--
Thanks,
Badai