You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by 黃竣陽 <s7...@gmail.com> on 2025/09/29 11:49:46 UTC
Re: [DISCUSS] KIP-1203: Allow to configure custom `ReplicaPlacer` implementation
Hi all,
I’d like to manually bump this thread. If anyone has feedback, please let me know.
Best Regards,
Jiunn-Yang
> 黃竣陽 <s7...@gmail.com> 於 2025年8月23日 下午4:24 寫道：
>
> Hello chia,
>
> Thanks for the reply, I have addressed all the comments.
>
> Best Regards,
> Jiunn-Yang
>
>> Chia-Ping Tsai <ch...@gmail.com> 於 2025年8月22日 中午12:20 寫道：
>>
>> hi Jiunn-Yang
>>
>>
>> chia_03
>> If the configuration is public, the package needs the package-info.
>>
>> chia_04
>> Additionally, the package includes some non-public stuff, which should be moved to another package. For example, StripedReplicaPlacer.
>>
>> chia_05
>> Please list the classes needing the UNSTABLE flag
>>
>> Best,
>> Chia-Ping
>>
>>
>>> 黃竣陽 <s7...@gmail.com> 於 2025年8月20日 晚上8:29 寫道：
>>>
>>> Hello chia, peng,
>>>
>>> I completely agree that Apache Kafka places a strong emphasis on compatibility.
>>> I believe the ReplicaPlacer plugin is primarily intended for advanced users—most
>>> common users are unlikely to implement their own plugin. Therefore, using an
>>> UNSTABLE or internal configuration is a trade-off we need to consider.
>>>
>>> If we make this configuration public and mark its interface as UNSTABLE, the
>>> advantage is that more users will be able to use it if they are comfortable with the API’s instability.
>>> The downside is that this would be the first UNSTABLE API exposed directly to users.
>>>
>>> Alternatively, if we make the configuration internal, the advantage is that we are not required to
>>> maintain compatibility since it is not exposed to users. However, the downside is that most users would
>>> not be aware of the configuration at all.
>>>
>>> On balance, I think making this configuration public (while clearly marking it as UNSTABLE) will provide
>>> more opportunities to gather feedback from users and improve the interface over time. Therefore, I will
>>> update this KIP to make the configuration public and mark it as UNSTABLE.
>>>
>>> Best Regards,
>>> Jiunn-Yang
>>>
>>>> peng <p1...@gmail.com> 於 2025年8月20日 下午4:13 寫道：
>>>> +1
>>>> best regards
>>>> Chia-Ping Tsai <ch...@apache.org> 于 2025年8月20日周三 下午3:46写道：
>>>>> I really like the idea of offering more flexible and powerful assignment
>>>>> policies. However, maintaining such implementations could become an
>>>>> overhead for the community. This is especially important because Apache
>>>>> Kafka places a strong emphasis on compatibility.
>>>>> The value of this KIP lies in opening the door for advanced developers to
>>>>> deploy custom assignments without modifying the source code.
>>>>> With respect to this KIP, perhaps we could expose a public configuration
>>>>> backed by UNSTABLE APIs. This approach would give us the flexibility to
>>>>> adjust the APIs in minor releases
>>>>> Best,
>>>>> Chia-Ping
>>>>>> On 2025/08/18 02:21:15 peng wrote:
>>>>>>> I think making `ReplicaPlacer` configurable is a good strategy. We can
>>>>>>> integrate KIP-1194: Optimize Replica Assignment for Broker Load Balance
>>>>> in
>>>>>> Uneven Rack Configurations into it, allowing users to choose between the
>>>>>> current `StripedReplicaPlacer` approach or the `WeightedReplicaPlacer`
>>>>>> mentioned in KIP-1194.
>>>>>> Perhaps the replica reassignment algorithm could also be made
>>>>> configurable,
>>>>>> enabling users to select solutions like KIP-1151: Minimal movement
>>>>> replica
>>>>>> balancing algorithm through configuration.
>>>>>> 黃竣陽 <s7...@gmail.com> 于 2025年8月6日周三 下午8:03写道：
>>>>>>> Hello chia,
>>>>>>> Thanks for the reply,
>>>>>>> chia_00, chia_01, chia_02: I have updated the KIP based on your
>>>>> comments.
>>>>>>> Best Regards,
>>>>>>> Jiunn-Yang
>>>>>>>> Chia-Ping Tsai <ch...@gmail.com> 於 2025年8月6日 凌晨12:18 寫道：
>>>>>>>> hi Jiunn-Yang
>>>>>>>> chia_00:
>>>>>>>> Could you please add more alternatives? For example, users could use
>>>>> the
>>>>>>>> admin to reassign partitions after the topics are created, or
>>>>> specify the
>>>>>>>> assignments directly when creating the topics
>>>>>>>> chia_01:
>>>>>>>> should we consider making `ReplicaPlacer` configurable?
>>>>>>>> chia_02:
>>>>>>>> Could you please enrice the documentation to remind users that this
>>>>>>>> configuration is internal, and clarify the specific use cases where
>>>>> it
>>>>>>>> might be relevant?
>>>>>>>> Best,
>>>>>>>> Chia-Ping
>>>>>>>> 黃竣陽 <s7...@gmail.com> 於 2025年8月5日 週二 下午11:24寫道：
>>>>>>>>> Hello everyone,
>>>>>>>>> I would like to start a discussion on KIP-1203: Allow to configure
>>>>>>> custom
>>>>>>>>> `ReplicaPlacer` implementation
>>>>>>>>> <https://cwiki.apache.org/confluence/x/dxFJFg>
>>>>>>>>> This proposal introduces a new internal configuration,
>>>>>>>>> replica.placer.class.name, which
>>>>>>>>> allows users to specify a custom implementation of ReplicaPlacer.
>>>>>>>>> Best regards,
>>>>>>>>> Jiunn-Yang
>
Re: [DISCUSS] KIP-1203: Allow to configure custom `ReplicaPlacer` implementation
Posted by 黃竣陽 <s7...@gmail.com>.
Hi all,
I’d like to manually bump this thread.
Best Regards,
Jiunn-Yang
黃竣陽 <s7...@gmail.com> 於 2025年9月29日 週一 下午7:49寫道：
> Hi all,
>
> I’d like to manually bump this thread. If anyone has feedback, please let
> me know.
>
> Best Regards,
> Jiunn-Yang
>
> > 黃竣陽 <s7...@gmail.com> 於 2025年8月23日 下午4:24 寫道：
> >
> > Hello chia,
> >
> > Thanks for the reply, I have addressed all the comments.
> >
> > Best Regards,
> > Jiunn-Yang
> >
> >> Chia-Ping Tsai <ch...@gmail.com> 於 2025年8月22日 中午12:20 寫道：
> >>
> >> hi Jiunn-Yang
> >>
> >>
> >> chia_03
> >> If the configuration is public, the package needs the package-info.
> >>
> >> chia_04
> >> Additionally, the package includes some non-public stuff, which should
> be moved to another package. For example, StripedReplicaPlacer.
> >>
> >> chia_05
> >> Please list the classes needing the UNSTABLE flag
> >>
> >> Best,
> >> Chia-Ping
> >>
> >>
> >>> 黃竣陽 <s7...@gmail.com> 於 2025年8月20日 晚上8:29 寫道：
> >>>
> >>> Hello chia, peng,
> >>>
> >>> I completely agree that Apache Kafka places a strong emphasis on
> compatibility.
> >>> I believe the ReplicaPlacer plugin is primarily intended for advanced
> users—most
> >>> common users are unlikely to implement their own plugin. Therefore,
> using an
> >>> UNSTABLE or internal configuration is a trade-off we need to consider.
> >>>
> >>> If we make this configuration public and mark its interface as
> UNSTABLE, the
> >>> advantage is that more users will be able to use it if they are
> comfortable with the API’s instability.
> >>> The downside is that this would be the first UNSTABLE API exposed
> directly to users.
> >>>
> >>> Alternatively, if we make the configuration internal, the advantage is
> that we are not required to
> >>> maintain compatibility since it is not exposed to users. However, the
> downside is that most users would
> >>> not be aware of the configuration at all.
> >>>
> >>> On balance, I think making this configuration public (while clearly
> marking it as UNSTABLE) will provide
> >>> more opportunities to gather feedback from users and improve the
> interface over time. Therefore, I will
> >>> update this KIP to make the configuration public and mark it as
> UNSTABLE.
> >>>
> >>> Best Regards,
> >>> Jiunn-Yang
> >>>
> >>>> peng <p1...@gmail.com> 於 2025年8月20日 下午4:13 寫道：
> >>>> +1
> >>>> best regards
> >>>> Chia-Ping Tsai <ch...@apache.org> 于 2025年8月20日周三 下午3:46写道：
> >>>>> I really like the idea of offering more flexible and powerful
> assignment
> >>>>> policies. However, maintaining such implementations could become an
> >>>>> overhead for the community. This is especially important because
> Apache
> >>>>> Kafka places a strong emphasis on compatibility.
> >>>>> The value of this KIP lies in opening the door for advanced
> developers to
> >>>>> deploy custom assignments without modifying the source code.
> >>>>> With respect to this KIP, perhaps we could expose a public
> configuration
> >>>>> backed by UNSTABLE APIs. This approach would give us the flexibility
> to
> >>>>> adjust the APIs in minor releases
> >>>>> Best,
> >>>>> Chia-Ping
> >>>>>> On 2025/08/18 02:21:15 peng wrote:
> >>>>>>> I think making `ReplicaPlacer` configurable is a good strategy. We
> can
> >>>>>>> integrate KIP-1194: Optimize Replica Assignment for Broker Load
> Balance
> >>>>> in
> >>>>>> Uneven Rack Configurations into it, allowing users to choose
> between the
> >>>>>> current `StripedReplicaPlacer` approach or the
> `WeightedReplicaPlacer`
> >>>>>> mentioned in KIP-1194.
> >>>>>> Perhaps the replica reassignment algorithm could also be made
> >>>>> configurable,
> >>>>>> enabling users to select solutions like KIP-1151: Minimal movement
> >>>>> replica
> >>>>>> balancing algorithm through configuration.
> >>>>>> 黃竣陽 <s7...@gmail.com> 于 2025年8月6日周三 下午8:03写道：
> >>>>>>> Hello chia,
> >>>>>>> Thanks for the reply,
> >>>>>>> chia_00, chia_01, chia_02: I have updated the KIP based on your
> >>>>> comments.
> >>>>>>> Best Regards,
> >>>>>>> Jiunn-Yang
> >>>>>>>> Chia-Ping Tsai <ch...@gmail.com> 於 2025年8月6日 凌晨12:18 寫道：
> >>>>>>>> hi Jiunn-Yang
> >>>>>>>> chia_00:
> >>>>>>>> Could you please add more alternatives? For example, users could
> use
> >>>>> the
> >>>>>>>> admin to reassign partitions after the topics are created, or
> >>>>> specify the
> >>>>>>>> assignments directly when creating the topics
> >>>>>>>> chia_01:
> >>>>>>>> should we consider making `ReplicaPlacer` configurable?
> >>>>>>>> chia_02:
> >>>>>>>> Could you please enrice the documentation to remind users that
> this
> >>>>>>>> configuration is internal, and clarify the specific use cases
> where
> >>>>> it
> >>>>>>>> might be relevant?
> >>>>>>>> Best,
> >>>>>>>> Chia-Ping
> >>>>>>>> 黃竣陽 <s7...@gmail.com> 於 2025年8月5日 週二 下午11:24寫道：
> >>>>>>>>> Hello everyone,
> >>>>>>>>> I would like to start a discussion on KIP-1203: Allow to
> configure
> >>>>>>> custom
> >>>>>>>>> `ReplicaPlacer` implementation
> >>>>>>>>> <https://cwiki.apache.org/confluence/x/dxFJFg>
> >>>>>>>>> This proposal introduces a new internal configuration,
> >>>>>>>>> replica.placer.class.name, which
> >>>>>>>>> allows users to specify a custom implementation of ReplicaPlacer.
> >>>>>>>>> Best regards,
> >>>>>>>>> Jiunn-Yang
> >
>
>