You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Colin McCabe <cm...@apache.org> on 2025/04/14 20:10:11 UTC
Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer
I would suggest adding a configuration key which controls whether the null values are added. That configuration key can default to true in 4.x and false in 5.x. This will give people a chance to test the new behavior before 5.x.
best,
Colin
On Fri, Mar 14, 2025, at 04:30, 黃竣陽 wrote:
> Hello everyone,
>
> I would like to start a discussion on KIP-1140: Avoid to return null
> value in Map from public api of consumer
> <https://cwiki.apache.org/confluence/x/mIuMEw>
>
> This proposal aims to improve the Kafka consumer API by ensuring that
> the Map it returns contains only non-null values,
> aligning with the design philosophy of Java collections. This change
> provides significantly more benefits than drawbacks,
> enhancing API consistency and usability while reducing errors caused by
> developer misuse.
>
> Best Regards,
> Jiunn-Yang
Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer
Posted by Kirk True <ki...@kirktrue.pro>.
Hi Chia-Ping,
An "enable unreleased behavior" flag is an interesting idea; basically a feature flag. The main benefit (coarsely grained) is also its biggest weakness. I'm not really sure the target audience that would enable it.
Thanks,
Kirk
On Mon, Apr 14, 2025, at 9:39 PM, Chia-Ping Tsai wrote:
> hi Colin
>
> Adding the config is a cool idea. However, the config will force us to maintain both behaviors in 5.0. Additionally, we need a complete deprecation cycle to remove the config.
>
> Maybe another alternative is to introduce a flag called “enable.unrelased.behavior”, allowing users to “test” the new behavior belonging to next major release. The default value is always false, and we don’t need to deprecate it even if there is no new behavior.
>
> The benefit from “enable.unrelased.behavior” is we don’t need to maintain both behaviors in 5.0. Additionally, the new config can serve for other similar behavior changes.
>
> Best,
> Chia-Ping
>
>
> > Colin McCabe <cm...@apache.org> 於 2025年4月15日 凌晨4:12 寫道：
> >
> > I would suggest adding a configuration key which controls whether the null values are added. That configuration key can default to true in 4.x and false in 5.x. This will give people a chance to test the new behavior before 5.x.
> >
> > best,
> > Colin
> >
> >> On Fri, Mar 14, 2025, at 04:30, 黃竣陽 wrote:
> >> Hello everyone,
> >>
> >> I would like to start a discussion on KIP-1140: Avoid to return null
> >> value in Map from public api of consumer
> >> <https://cwiki.apache.org/confluence/x/mIuMEw>
> >>
> >> This proposal aims to improve the Kafka consumer API by ensuring that
> >> the Map it returns contains only non-null values,
> >> aligning with the design philosophy of Java collections. This change
> >> provides significantly more benefits than drawbacks,
> >> enhancing API consistency and usability while reducing errors caused by
> >> developer misuse.
> >>
> >> Best Regards,
> >> Jiunn-Yang
>
Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer
Posted by Chia-Ping Tsai <ch...@gmail.com>.
hi Colin
Adding the config is a cool idea. However, the config will force us to maintain both behaviors in 5.0. Additionally, we need a complete deprecation cycle to remove the config.
Maybe another alternative is to introduce a flag called “enable.unrelased.behavior”, allowing users to “test” the new behavior belonging to next major release. The default value is always false, and we don’t need to deprecate it even if there is no new behavior.
The benefit from “enable.unrelased.behavior” is we don’t need to maintain both behaviors in 5.0. Additionally, the new config can serve for other similar behavior changes.
Best,
Chia-Ping
> Colin McCabe <cm...@apache.org> 於 2025年4月15日 凌晨4:12 寫道：
>
> I would suggest adding a configuration key which controls whether the null values are added. That configuration key can default to true in 4.x and false in 5.x. This will give people a chance to test the new behavior before 5.x.
>
> best,
> Colin
>
>> On Fri, Mar 14, 2025, at 04:30, 黃竣陽 wrote:
>> Hello everyone,
>>
>> I would like to start a discussion on KIP-1140: Avoid to return null
>> value in Map from public api of consumer
>> <https://cwiki.apache.org/confluence/x/mIuMEw>
>>
>> This proposal aims to improve the Kafka consumer API by ensuring that
>> the Map it returns contains only non-null values,
>> aligning with the design philosophy of Java collections. This change
>> provides significantly more benefits than drawbacks,
>> enhancing API consistency and usability while reducing errors caused by
>> developer misuse.
>>
>> Best Regards,
>> Jiunn-Yang
Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer
Posted by 黃竣陽 <s7...@gmail.com>.
Hi Kirk,
Thanks for your comment.
I agree that ‘allow.null.offsets.entries' is a better name. I've updated the KIP accordingly.
Best Regards,
Jiunn-Yang
> Kirk True <ki...@kirktrue.pro> 於 2025年4月23日 上午8:20 寫道：
>
> Hi Jiunn-Yang,
>
> Thanks for the updates!
>
> IMO, "allow.null.entries" might be a little too vague. Is "allow.null.offsets.entries" better?
>
> Thanks,
> Kirk
>
> On Tue, Apr 22, 2025, at 6:03 AM, 黃竣陽 wrote:
>> Hello everyone,
>>
>> I’ve submitted an updated version of this KIP based on recent discussions. I'm happy to hear any further concerns
>> or suggestions you might have.
>>
>> <https://cwiki.apache.org/confluence/x/mIuMEw>
>>
>> Best Regards,
>> Jiunn-Yang
>>
>>> 黃竣陽 <s7...@gmail.com> 於 2025年4月22日 下午6:35 寫道：
>>>
>>> Hello,
>>>
>>> Thank for the feedback. I will address this new/deprecated mechanism in the updated version of the KIP.
>>>
>>> Best Regards,
>>> Jiunn-Yang
>>>
>>>> Chia-Ping Tsai <ch...@gmail.com> 於 2025年4月22日 中午12:58 寫道：
>>>>
>>>> hi Matthias
>>>>
>>>> Thanks for offering this approach. I had considered proposing it before. My concern was the new/deprecated config is used by client, so it could cause a bunch of warning messages by default. Also, users have to add the new/deprecated config to each client properties if they want to eliminate the noise.
>>>>
>>>> However, as Juma mentioned that the approach was applied by another KIP before, so +1 to the new/deprecated config.
>>>>
>>>> To Ken
>>>>
>>>> Could you please update the KIP to include the new config? Additionally, please add the other discussed approaches to the “rejected” section.
>>>>
>>>> Best,
>>>> Chia-Ping
>>>>
>>>>
>>>>
>>>>
>>>>> Ismael Juma <me...@ismaeljuma.com.invalid> 於 2025年4月22日 上午11:03 寫道：
>>>>>
>>>>> That's right. Similar approach:
>>>>> https://lists.apache.org/thread/3fxqdsosm3z7xp4rr8db2qmyg5vd0v1b
>>>>>
>>>>> Ismael
>>>>>
>>>>>> On Mon, Apr 21, 2025, 7:43 PM Matthias J. Sax <mj...@apache.org> wrote:
>>>>>>
>>>>>> I don't think we would need a second deprecation cycle.
>>>>>>
>>>>>> Instead, we could add this new config, and deprecate it right away. This
>>>>>> way, we can change the default behavior and remove the config both with
>>>>>> 5.0.
>>>>>>
>>>>>> I guess the core question is really, do we want to enable the old or new
>>>>>> behavior by default? From a backward compatibility POV, we would need to
>>>>>> keep the old behavior as default IMHO.
>>>>>>
>>>>>> If we can agree to this, the way forward might be:
>>>>>>
>>>>>> - keep the old behavior as default
>>>>>> - add a config that allows to enable the new behavior
>>>>>> - if use don't enable the new behavior, log a WARN telling them that
>>>>>> they need to migrate their code before 5.0 and encourage them to enable
>>>>>> the new behavior right away
>>>>>> - deprecate the new config right away
>>>>>> - remove the config and old behavior in 5.0
>>>>>>
>>>>>>
>>>>>> I admit, that adding a new config, and deprecating it, plus telling
>>>>>> users "please use this new/deprecated config", is a little bit awkward
>>>>>> -- but it seems to be still the overall best way forward IMHO?
>>>>>>
>>>>>>
>>>>>>
>>>>>> -Matthias
>>>>>>
>>>>>>> On 4/17/25 8:16 PM, Chia-Ping Tsai wrote:
>>>>>>> hi Kirk
>>>>>>>
>>>>>>> “coarsely grained” is a good point. We need to list all behaviors
>>>>>> impacted by the config - no matter which new config we adopted.
>>>>>>>
>>>>>>> Maybe we can add the flag to xxxOption? The benefit is users can
>>>>>> explicitly see “which” APIs are impacted. The downside is the number of
>>>>>> deprecated methods in 5.0 is increased
>>>>>>>
>>>>>>> Best,
>>>>>>> Chia-Ping
>>>>>>>
>>>>>>>> Kirk True <ki...@kirktrue.pro> 於 2025年4月18日 清晨7:16 寫道：
>>>>>>>>
>>>>>>>> Hi Colin,
>>>>>>>>
>>>>>>>> If so, is 5.0 the earliest this 'allow.nulls.in.consumer' configuration
>>>>>> can be changed and marked as deprecated? And if that holds, is 6.0 the
>>>>>> earliest it can be removed?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Kirk
>>>>>>>>
>>>>>>>>> On Mon, Apr 14, 2025, at 1:10 PM, Colin McCabe wrote:
>>>>>>>>> I would suggest adding a configuration key which controls whether the
>>>>>> null values are added. That configuration key can default to true in 4.x
>>>>>> and false in 5.x. This will give people a chance to test the new behavior
>>>>>> before 5.x.
>>>>>>>>> best,
>>>>>>>>> Colin
>>>>>>>>>> On Fri, Mar 14, 2025, at 04:30, 黃竣陽 wrote:
>>>>>>>>>> Hello everyone,
>>>>>>>>>> I would like to start a discussion on KIP-1140: Avoid to return null
>>>>>>>>>> value in Map from public api of consumer
>>>>>>>>>> <https://cwiki.apache.org/confluence/x/mIuMEw>
>>>>>>>>>> This proposal aims to improve the Kafka consumer API by ensuring that
>>>>>>>>>> the Map it returns contains only non-null values,
>>>>>>>>>> aligning with the design philosophy of Java collections. This change
>>>>>>>>>> provides significantly more benefits than drawbacks,
>>>>>>>>>> enhancing API consistency and usability while reducing errors caused
>>>>>> by
>>>>>>>>>> developer misuse.
>>>>>>>>>> Best Regards,
>>>>>>>>>> Jiunn-Yang
>>>>>>
>>>>>>
>>>
>>
>>
Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer
Posted by Kirk True <ki...@kirktrue.pro>.
Hi Jiunn-Yang,
Thanks for the updates!
IMO, "allow.null.entries" might be a little too vague. Is "allow.null.offsets.entries" better?
Thanks,
Kirk
On Tue, Apr 22, 2025, at 6:03 AM, 黃竣陽 wrote:
> Hello everyone,
>
> I’ve submitted an updated version of this KIP based on recent discussions. I'm happy to hear any further concerns
> or suggestions you might have.
>
> <https://cwiki.apache.org/confluence/x/mIuMEw>
>
> Best Regards,
> Jiunn-Yang
>
> > 黃竣陽 <s7...@gmail.com> 於 2025年4月22日 下午6:35 寫道：
> >
> > Hello,
> >
> > Thank for the feedback. I will address this new/deprecated mechanism in the updated version of the KIP.
> >
> > Best Regards,
> > Jiunn-Yang
> >
> >> Chia-Ping Tsai <ch...@gmail.com> 於 2025年4月22日 中午12:58 寫道：
> >>
> >> hi Matthias
> >>
> >> Thanks for offering this approach. I had considered proposing it before. My concern was the new/deprecated config is used by client, so it could cause a bunch of warning messages by default. Also, users have to add the new/deprecated config to each client properties if they want to eliminate the noise.
> >>
> >> However, as Juma mentioned that the approach was applied by another KIP before, so +1 to the new/deprecated config.
> >>
> >> To Ken
> >>
> >> Could you please update the KIP to include the new config? Additionally, please add the other discussed approaches to the “rejected” section.
> >>
> >> Best,
> >> Chia-Ping
> >>
> >>
> >>
> >>
> >>> Ismael Juma <me...@ismaeljuma.com.invalid> 於 2025年4月22日 上午11:03 寫道：
> >>>
> >>> That's right. Similar approach:
> >>> https://lists.apache.org/thread/3fxqdsosm3z7xp4rr8db2qmyg5vd0v1b
> >>>
> >>> Ismael
> >>>
> >>>> On Mon, Apr 21, 2025, 7:43 PM Matthias J. Sax <mj...@apache.org> wrote:
> >>>>
> >>>> I don't think we would need a second deprecation cycle.
> >>>>
> >>>> Instead, we could add this new config, and deprecate it right away. This
> >>>> way, we can change the default behavior and remove the config both with
> >>>> 5.0.
> >>>>
> >>>> I guess the core question is really, do we want to enable the old or new
> >>>> behavior by default? From a backward compatibility POV, we would need to
> >>>> keep the old behavior as default IMHO.
> >>>>
> >>>> If we can agree to this, the way forward might be:
> >>>>
> >>>> - keep the old behavior as default
> >>>> - add a config that allows to enable the new behavior
> >>>> - if use don't enable the new behavior, log a WARN telling them that
> >>>> they need to migrate their code before 5.0 and encourage them to enable
> >>>> the new behavior right away
> >>>> - deprecate the new config right away
> >>>> - remove the config and old behavior in 5.0
> >>>>
> >>>>
> >>>> I admit, that adding a new config, and deprecating it, plus telling
> >>>> users "please use this new/deprecated config", is a little bit awkward
> >>>> -- but it seems to be still the overall best way forward IMHO?
> >>>>
> >>>>
> >>>>
> >>>> -Matthias
> >>>>
> >>>>> On 4/17/25 8:16 PM, Chia-Ping Tsai wrote:
> >>>>> hi Kirk
> >>>>>
> >>>>> “coarsely grained” is a good point. We need to list all behaviors
> >>>> impacted by the config - no matter which new config we adopted.
> >>>>>
> >>>>> Maybe we can add the flag to xxxOption? The benefit is users can
> >>>> explicitly see “which” APIs are impacted. The downside is the number of
> >>>> deprecated methods in 5.0 is increased
> >>>>>
> >>>>> Best,
> >>>>> Chia-Ping
> >>>>>
> >>>>>> Kirk True <ki...@kirktrue.pro> 於 2025年4月18日 清晨7:16 寫道：
> >>>>>>
> >>>>>> Hi Colin,
> >>>>>>
> >>>>>> If so, is 5.0 the earliest this 'allow.nulls.in.consumer' configuration
> >>>> can be changed and marked as deprecated? And if that holds, is 6.0 the
> >>>> earliest it can be removed?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Kirk
> >>>>>>
> >>>>>>> On Mon, Apr 14, 2025, at 1:10 PM, Colin McCabe wrote:
> >>>>>>> I would suggest adding a configuration key which controls whether the
> >>>> null values are added. That configuration key can default to true in 4.x
> >>>> and false in 5.x. This will give people a chance to test the new behavior
> >>>> before 5.x.
> >>>>>>> best,
> >>>>>>> Colin
> >>>>>>>> On Fri, Mar 14, 2025, at 04:30, 黃竣陽 wrote:
> >>>>>>>> Hello everyone,
> >>>>>>>> I would like to start a discussion on KIP-1140: Avoid to return null
> >>>>>>>> value in Map from public api of consumer
> >>>>>>>> <https://cwiki.apache.org/confluence/x/mIuMEw>
> >>>>>>>> This proposal aims to improve the Kafka consumer API by ensuring that
> >>>>>>>> the Map it returns contains only non-null values,
> >>>>>>>> aligning with the design philosophy of Java collections. This change
> >>>>>>>> provides significantly more benefits than drawbacks,
> >>>>>>>> enhancing API consistency and usability while reducing errors caused
> >>>> by
> >>>>>>>> developer misuse.
> >>>>>>>> Best Regards,
> >>>>>>>> Jiunn-Yang
> >>>>
> >>>>
> >
>
>
Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer
Posted by 黃竣陽 <s7...@gmail.com>.
Hello everyone,
I’ve submitted an updated version of this KIP based on recent discussions. I'm happy to hear any further concerns
or suggestions you might have.
<https://cwiki.apache.org/confluence/x/mIuMEw>
Best Regards,
Jiunn-Yang
> 黃竣陽 <s7...@gmail.com> 於 2025年4月22日 下午6:35 寫道：
>
> Hello,
>
> Thank for the feedback. I will address this new/deprecated mechanism in the updated version of the KIP.
>
> Best Regards,
> Jiunn-Yang
>
>> Chia-Ping Tsai <ch...@gmail.com> 於 2025年4月22日 中午12:58 寫道：
>>
>> hi Matthias
>>
>> Thanks for offering this approach. I had considered proposing it before. My concern was the new/deprecated config is used by client, so it could cause a bunch of warning messages by default. Also, users have to add the new/deprecated config to each client properties if they want to eliminate the noise.
>>
>> However, as Juma mentioned that the approach was applied by another KIP before, so +1 to the new/deprecated config.
>>
>> To Ken
>>
>> Could you please update the KIP to include the new config? Additionally, please add the other discussed approaches to the “rejected” section.
>>
>> Best,
>> Chia-Ping
>>
>>
>>
>>
>>> Ismael Juma <me...@ismaeljuma.com.invalid> 於 2025年4月22日 上午11:03 寫道：
>>>
>>> That's right. Similar approach:
>>> https://lists.apache.org/thread/3fxqdsosm3z7xp4rr8db2qmyg5vd0v1b
>>>
>>> Ismael
>>>
>>>> On Mon, Apr 21, 2025, 7:43 PM Matthias J. Sax <mj...@apache.org> wrote:
>>>>
>>>> I don't think we would need a second deprecation cycle.
>>>>
>>>> Instead, we could add this new config, and deprecate it right away. This
>>>> way, we can change the default behavior and remove the config both with
>>>> 5.0.
>>>>
>>>> I guess the core question is really, do we want to enable the old or new
>>>> behavior by default? From a backward compatibility POV, we would need to
>>>> keep the old behavior as default IMHO.
>>>>
>>>> If we can agree to this, the way forward might be:
>>>>
>>>> - keep the old behavior as default
>>>> - add a config that allows to enable the new behavior
>>>> - if use don't enable the new behavior, log a WARN telling them that
>>>> they need to migrate their code before 5.0 and encourage them to enable
>>>> the new behavior right away
>>>> - deprecate the new config right away
>>>> - remove the config and old behavior in 5.0
>>>>
>>>>
>>>> I admit, that adding a new config, and deprecating it, plus telling
>>>> users "please use this new/deprecated config", is a little bit awkward
>>>> -- but it seems to be still the overall best way forward IMHO?
>>>>
>>>>
>>>>
>>>> -Matthias
>>>>
>>>>> On 4/17/25 8:16 PM, Chia-Ping Tsai wrote:
>>>>> hi Kirk
>>>>>
>>>>> “coarsely grained” is a good point. We need to list all behaviors
>>>> impacted by the config - no matter which new config we adopted.
>>>>>
>>>>> Maybe we can add the flag to xxxOption? The benefit is users can
>>>> explicitly see “which” APIs are impacted. The downside is the number of
>>>> deprecated methods in 5.0 is increased
>>>>>
>>>>> Best,
>>>>> Chia-Ping
>>>>>
>>>>>> Kirk True <ki...@kirktrue.pro> 於 2025年4月18日 清晨7:16 寫道：
>>>>>>
>>>>>> Hi Colin,
>>>>>>
>>>>>> If so, is 5.0 the earliest this 'allow.nulls.in.consumer' configuration
>>>> can be changed and marked as deprecated? And if that holds, is 6.0 the
>>>> earliest it can be removed?
>>>>>>
>>>>>> Thanks,
>>>>>> Kirk
>>>>>>
>>>>>>> On Mon, Apr 14, 2025, at 1:10 PM, Colin McCabe wrote:
>>>>>>> I would suggest adding a configuration key which controls whether the
>>>> null values are added. That configuration key can default to true in 4.x
>>>> and false in 5.x. This will give people a chance to test the new behavior
>>>> before 5.x.
>>>>>>> best,
>>>>>>> Colin
>>>>>>>> On Fri, Mar 14, 2025, at 04:30, 黃竣陽 wrote:
>>>>>>>> Hello everyone,
>>>>>>>> I would like to start a discussion on KIP-1140: Avoid to return null
>>>>>>>> value in Map from public api of consumer
>>>>>>>> <https://cwiki.apache.org/confluence/x/mIuMEw>
>>>>>>>> This proposal aims to improve the Kafka consumer API by ensuring that
>>>>>>>> the Map it returns contains only non-null values,
>>>>>>>> aligning with the design philosophy of Java collections. This change
>>>>>>>> provides significantly more benefits than drawbacks,
>>>>>>>> enhancing API consistency and usability while reducing errors caused
>>>> by
>>>>>>>> developer misuse.
>>>>>>>> Best Regards,
>>>>>>>> Jiunn-Yang
>>>>
>>>>
>
Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer
Posted by 黃竣陽 <s7...@gmail.com>.
Hello,
Thank for the feedback. I will address this new/deprecated mechanism in the updated version of the KIP.
Best Regards,
Jiunn-Yang
> Chia-Ping Tsai <ch...@gmail.com> 於 2025年4月22日 中午12:58 寫道：
>
> hi Matthias
>
> Thanks for offering this approach. I had considered proposing it before. My concern was the new/deprecated config is used by client, so it could cause a bunch of warning messages by default. Also, users have to add the new/deprecated config to each client properties if they want to eliminate the noise.
>
> However, as Juma mentioned that the approach was applied by another KIP before, so +1 to the new/deprecated config.
>
> To Ken
>
> Could you please update the KIP to include the new config? Additionally, please add the other discussed approaches to the “rejected” section.
>
> Best,
> Chia-Ping
>
>
>
>
>> Ismael Juma <me...@ismaeljuma.com.invalid> 於 2025年4月22日 上午11:03 寫道：
>>
>> That's right. Similar approach:
>> https://lists.apache.org/thread/3fxqdsosm3z7xp4rr8db2qmyg5vd0v1b
>>
>> Ismael
>>
>>> On Mon, Apr 21, 2025, 7:43 PM Matthias J. Sax <mj...@apache.org> wrote:
>>>
>>> I don't think we would need a second deprecation cycle.
>>>
>>> Instead, we could add this new config, and deprecate it right away. This
>>> way, we can change the default behavior and remove the config both with
>>> 5.0.
>>>
>>> I guess the core question is really, do we want to enable the old or new
>>> behavior by default? From a backward compatibility POV, we would need to
>>> keep the old behavior as default IMHO.
>>>
>>> If we can agree to this, the way forward might be:
>>>
>>> - keep the old behavior as default
>>> - add a config that allows to enable the new behavior
>>> - if use don't enable the new behavior, log a WARN telling them that
>>> they need to migrate their code before 5.0 and encourage them to enable
>>> the new behavior right away
>>> - deprecate the new config right away
>>> - remove the config and old behavior in 5.0
>>>
>>>
>>> I admit, that adding a new config, and deprecating it, plus telling
>>> users "please use this new/deprecated config", is a little bit awkward
>>> -- but it seems to be still the overall best way forward IMHO?
>>>
>>>
>>>
>>> -Matthias
>>>
>>>> On 4/17/25 8:16 PM, Chia-Ping Tsai wrote:
>>>> hi Kirk
>>>>
>>>> “coarsely grained” is a good point. We need to list all behaviors
>>> impacted by the config - no matter which new config we adopted.
>>>>
>>>> Maybe we can add the flag to xxxOption? The benefit is users can
>>> explicitly see “which” APIs are impacted. The downside is the number of
>>> deprecated methods in 5.0 is increased
>>>>
>>>> Best,
>>>> Chia-Ping
>>>>
>>>>> Kirk True <ki...@kirktrue.pro> 於 2025年4月18日 清晨7:16 寫道：
>>>>>
>>>>> Hi Colin,
>>>>>
>>>>> If so, is 5.0 the earliest this 'allow.nulls.in.consumer' configuration
>>> can be changed and marked as deprecated? And if that holds, is 6.0 the
>>> earliest it can be removed?
>>>>>
>>>>> Thanks,
>>>>> Kirk
>>>>>
>>>>>> On Mon, Apr 14, 2025, at 1:10 PM, Colin McCabe wrote:
>>>>>> I would suggest adding a configuration key which controls whether the
>>> null values are added. That configuration key can default to true in 4.x
>>> and false in 5.x. This will give people a chance to test the new behavior
>>> before 5.x.
>>>>>> best,
>>>>>> Colin
>>>>>>> On Fri, Mar 14, 2025, at 04:30, 黃竣陽 wrote:
>>>>>>> Hello everyone,
>>>>>>> I would like to start a discussion on KIP-1140: Avoid to return null
>>>>>>> value in Map from public api of consumer
>>>>>>> <https://cwiki.apache.org/confluence/x/mIuMEw>
>>>>>>> This proposal aims to improve the Kafka consumer API by ensuring that
>>>>>>> the Map it returns contains only non-null values,
>>>>>>> aligning with the design philosophy of Java collections. This change
>>>>>>> provides significantly more benefits than drawbacks,
>>>>>>> enhancing API consistency and usability while reducing errors caused
>>> by
>>>>>>> developer misuse.
>>>>>>> Best Regards,
>>>>>>> Jiunn-Yang
>>>
>>>
Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer
Posted by Chia-Ping Tsai <ch...@gmail.com>.
hi Matthias
Thanks for offering this approach. I had considered proposing it before. My concern was the new/deprecated config is used by client, so it could cause a bunch of warning messages by default. Also, users have to add the new/deprecated config to each client properties if they want to eliminate the noise.
However, as Juma mentioned that the approach was applied by another KIP before, so +1 to the new/deprecated config.
To Ken
Could you please update the KIP to include the new config? Additionally, please add the other discussed approaches to the “rejected” section.
Best,
Chia-Ping
> Ismael Juma <me...@ismaeljuma.com.invalid> 於 2025年4月22日 上午11:03 寫道：
>
> That's right. Similar approach:
> https://lists.apache.org/thread/3fxqdsosm3z7xp4rr8db2qmyg5vd0v1b
>
> Ismael
>
>> On Mon, Apr 21, 2025, 7:43 PM Matthias J. Sax <mj...@apache.org> wrote:
>>
>> I don't think we would need a second deprecation cycle.
>>
>> Instead, we could add this new config, and deprecate it right away. This
>> way, we can change the default behavior and remove the config both with
>> 5.0.
>>
>> I guess the core question is really, do we want to enable the old or new
>> behavior by default? From a backward compatibility POV, we would need to
>> keep the old behavior as default IMHO.
>>
>> If we can agree to this, the way forward might be:
>>
>> - keep the old behavior as default
>> - add a config that allows to enable the new behavior
>> - if use don't enable the new behavior, log a WARN telling them that
>> they need to migrate their code before 5.0 and encourage them to enable
>> the new behavior right away
>> - deprecate the new config right away
>> - remove the config and old behavior in 5.0
>>
>>
>> I admit, that adding a new config, and deprecating it, plus telling
>> users "please use this new/deprecated config", is a little bit awkward
>> -- but it seems to be still the overall best way forward IMHO?
>>
>>
>>
>> -Matthias
>>
>>> On 4/17/25 8:16 PM, Chia-Ping Tsai wrote:
>>> hi Kirk
>>>
>>> “coarsely grained” is a good point. We need to list all behaviors
>> impacted by the config - no matter which new config we adopted.
>>>
>>> Maybe we can add the flag to xxxOption? The benefit is users can
>> explicitly see “which” APIs are impacted. The downside is the number of
>> deprecated methods in 5.0 is increased
>>>
>>> Best,
>>> Chia-Ping
>>>
>>>> Kirk True <ki...@kirktrue.pro> 於 2025年4月18日 清晨7:16 寫道：
>>>>
>>>> Hi Colin,
>>>>
>>>> If so, is 5.0 the earliest this 'allow.nulls.in.consumer' configuration
>> can be changed and marked as deprecated? And if that holds, is 6.0 the
>> earliest it can be removed?
>>>>
>>>> Thanks,
>>>> Kirk
>>>>
>>>>> On Mon, Apr 14, 2025, at 1:10 PM, Colin McCabe wrote:
>>>>> I would suggest adding a configuration key which controls whether the
>> null values are added. That configuration key can default to true in 4.x
>> and false in 5.x. This will give people a chance to test the new behavior
>> before 5.x.
>>>>> best,
>>>>> Colin
>>>>>> On Fri, Mar 14, 2025, at 04:30, 黃竣陽 wrote:
>>>>>> Hello everyone,
>>>>>> I would like to start a discussion on KIP-1140: Avoid to return null
>>>>>> value in Map from public api of consumer
>>>>>> <https://cwiki.apache.org/confluence/x/mIuMEw>
>>>>>> This proposal aims to improve the Kafka consumer API by ensuring that
>>>>>> the Map it returns contains only non-null values,
>>>>>> aligning with the design philosophy of Java collections. This change
>>>>>> provides significantly more benefits than drawbacks,
>>>>>> enhancing API consistency and usability while reducing errors caused
>> by
>>>>>> developer misuse.
>>>>>> Best Regards,
>>>>>> Jiunn-Yang
>>
>>
Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer
Posted by Ismael Juma <me...@ismaeljuma.com.INVALID>.
That's right. Similar approach:
https://lists.apache.org/thread/3fxqdsosm3z7xp4rr8db2qmyg5vd0v1b
Ismael
On Mon, Apr 21, 2025, 7:43 PM Matthias J. Sax <mj...@apache.org> wrote:
> I don't think we would need a second deprecation cycle.
>
> Instead, we could add this new config, and deprecate it right away. This
> way, we can change the default behavior and remove the config both with
> 5.0.
>
> I guess the core question is really, do we want to enable the old or new
> behavior by default? From a backward compatibility POV, we would need to
> keep the old behavior as default IMHO.
>
> If we can agree to this, the way forward might be:
>
> - keep the old behavior as default
> - add a config that allows to enable the new behavior
> - if use don't enable the new behavior, log a WARN telling them that
> they need to migrate their code before 5.0 and encourage them to enable
> the new behavior right away
> - deprecate the new config right away
> - remove the config and old behavior in 5.0
>
>
> I admit, that adding a new config, and deprecating it, plus telling
> users "please use this new/deprecated config", is a little bit awkward
> -- but it seems to be still the overall best way forward IMHO?
>
>
>
> -Matthias
>
> On 4/17/25 8:16 PM, Chia-Ping Tsai wrote:
> > hi Kirk
> >
> > “coarsely grained” is a good point. We need to list all behaviors
> impacted by the config - no matter which new config we adopted.
> >
> > Maybe we can add the flag to xxxOption? The benefit is users can
> explicitly see “which” APIs are impacted. The downside is the number of
> deprecated methods in 5.0 is increased
> >
> > Best,
> > Chia-Ping
> >
> >> Kirk True <ki...@kirktrue.pro> 於 2025年4月18日 清晨7:16 寫道：
> >>
> >> Hi Colin,
> >>
> >> If so, is 5.0 the earliest this 'allow.nulls.in.consumer' configuration
> can be changed and marked as deprecated? And if that holds, is 6.0 the
> earliest it can be removed?
> >>
> >> Thanks,
> >> Kirk
> >>
> >>> On Mon, Apr 14, 2025, at 1:10 PM, Colin McCabe wrote:
> >>> I would suggest adding a configuration key which controls whether the
> null values are added. That configuration key can default to true in 4.x
> and false in 5.x. This will give people a chance to test the new behavior
> before 5.x.
> >>> best,
> >>> Colin
> >>>> On Fri, Mar 14, 2025, at 04:30, 黃竣陽 wrote:
> >>>> Hello everyone,
> >>>> I would like to start a discussion on KIP-1140: Avoid to return null
> >>>> value in Map from public api of consumer
> >>>> <https://cwiki.apache.org/confluence/x/mIuMEw>
> >>>> This proposal aims to improve the Kafka consumer API by ensuring that
> >>>> the Map it returns contains only non-null values,
> >>>> aligning with the design philosophy of Java collections. This change
> >>>> provides significantly more benefits than drawbacks,
> >>>> enhancing API consistency and usability while reducing errors caused
> by
> >>>> developer misuse.
> >>>> Best Regards,
> >>>> Jiunn-Yang
>
>
Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer
Posted by "Matthias J. Sax" <mj...@apache.org>.
I don't think we would need a second deprecation cycle.
Instead, we could add this new config, and deprecate it right away. This
way, we can change the default behavior and remove the config both with 5.0.
I guess the core question is really, do we want to enable the old or new
behavior by default? From a backward compatibility POV, we would need to
keep the old behavior as default IMHO.
If we can agree to this, the way forward might be:
- keep the old behavior as default
- add a config that allows to enable the new behavior
- if use don't enable the new behavior, log a WARN telling them that
they need to migrate their code before 5.0 and encourage them to enable
the new behavior right away
- deprecate the new config right away
- remove the config and old behavior in 5.0
I admit, that adding a new config, and deprecating it, plus telling
users "please use this new/deprecated config", is a little bit awkward
-- but it seems to be still the overall best way forward IMHO?
-Matthias
On 4/17/25 8:16 PM, Chia-Ping Tsai wrote:
> hi Kirk
>
> “coarsely grained” is a good point. We need to list all behaviors impacted by the config - no matter which new config we adopted.
>
> Maybe we can add the flag to xxxOption? The benefit is users can explicitly see “which” APIs are impacted. The downside is the number of deprecated methods in 5.0 is increased
>
> Best,
> Chia-Ping
>
>> Kirk True <ki...@kirktrue.pro> 於 2025年4月18日 清晨7:16 寫道：
>>
>> Hi Colin,
>>
>> If so, is 5.0 the earliest this 'allow.nulls.in.consumer' configuration can be changed and marked as deprecated? And if that holds, is 6.0 the earliest it can be removed?
>>
>> Thanks,
>> Kirk
>>
>>> On Mon, Apr 14, 2025, at 1:10 PM, Colin McCabe wrote:
>>> I would suggest adding a configuration key which controls whether the null values are added. That configuration key can default to true in 4.x and false in 5.x. This will give people a chance to test the new behavior before 5.x.
>>> best,
>>> Colin
>>>> On Fri, Mar 14, 2025, at 04:30, 黃竣陽 wrote:
>>>> Hello everyone,
>>>> I would like to start a discussion on KIP-1140: Avoid to return null
>>>> value in Map from public api of consumer
>>>> <https://cwiki.apache.org/confluence/x/mIuMEw>
>>>> This proposal aims to improve the Kafka consumer API by ensuring that
>>>> the Map it returns contains only non-null values,
>>>> aligning with the design philosophy of Java collections. This change
>>>> provides significantly more benefits than drawbacks,
>>>> enhancing API consistency and usability while reducing errors caused by
>>>> developer misuse.
>>>> Best Regards,
>>>> Jiunn-Yang
Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer
Posted by Chia-Ping Tsai <ch...@gmail.com>.
hi Kirk
“coarsely grained” is a good point. We need to list all behaviors impacted by the config - no matter which new config we adopted.
Maybe we can add the flag to xxxOption? The benefit is users can explicitly see “which” APIs are impacted. The downside is the number of deprecated methods in 5.0 is increased
Best,
Chia-Ping
> Kirk True <ki...@kirktrue.pro> 於 2025年4月18日 清晨7:16 寫道：
>
> Hi Colin,
>
> If so, is 5.0 the earliest this 'allow.nulls.in.consumer' configuration can be changed and marked as deprecated? And if that holds, is 6.0 the earliest it can be removed?
>
> Thanks,
> Kirk
>
>> On Mon, Apr 14, 2025, at 1:10 PM, Colin McCabe wrote:
>> I would suggest adding a configuration key which controls whether the null values are added. That configuration key can default to true in 4.x and false in 5.x. This will give people a chance to test the new behavior before 5.x.
>> best,
>> Colin
>>> On Fri, Mar 14, 2025, at 04:30, 黃竣陽 wrote:
>>> Hello everyone,
>>> I would like to start a discussion on KIP-1140: Avoid to return null
>>> value in Map from public api of consumer
>>> <https://cwiki.apache.org/confluence/x/mIuMEw>
>>> This proposal aims to improve the Kafka consumer API by ensuring that
>>> the Map it returns contains only non-null values,
>>> aligning with the design philosophy of Java collections. This change
>>> provides significantly more benefits than drawbacks,
>>> enhancing API consistency and usability while reducing errors caused by
>>> developer misuse.
>>> Best Regards,
>>> Jiunn-Yang
Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer
Posted by Kirk True <ki...@kirktrue.pro>.
Hi Colin,
If so, is 5.0 the earliest this 'allow.nulls.in.consumer' configuration can be changed and marked as deprecated? And if that holds, is 6.0 the earliest it can be removed?
Thanks,
Kirk
On Mon, Apr 14, 2025, at 1:10 PM, Colin McCabe wrote:
> I would suggest adding a configuration key which controls whether the null values are added. That configuration key can default to true in 4.x and false in 5.x. This will give people a chance to test the new behavior before 5.x.
>
> best,
> Colin
>
> On Fri, Mar 14, 2025, at 04:30, 黃竣陽 wrote:
> > Hello everyone,
> >
> > I would like to start a discussion on KIP-1140: Avoid to return null
> > value in Map from public api of consumer
> > <https://cwiki.apache.org/confluence/x/mIuMEw>
> >
> > This proposal aims to improve the Kafka consumer API by ensuring that
> > the Map it returns contains only non-null values,
> > aligning with the design philosophy of Java collections. This change
> > provides significantly more benefits than drawbacks,
> > enhancing API consistency and usability while reducing errors caused by
> > developer misuse.
> >
> > Best Regards,
> > Jiunn-Yang
>