You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Sergey Korytnikov <se...@gmail.com> on 2018/03/16 13:24:07 UTC
[DISCUSS] KIP-271 Add NetworkClient redirector
Hi devs,
I have created new KIP-271 for discussion:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-271%3A+Add+NetworkClient+redirector
Kafka JIRA issue: https://issues.apache.org/jira/browse/KAFKA-6669
The fix has been implemented and tested with version 1.0 and trunk snapshot
of clients jar.
Changes are available here:
https://github.com/ssbiox/kafka/tree/trunk-redirector
Diff of changes:
https://github.com/apache/kafka/compare/trunk...ssbiox:trunk-redirector
Thank you,
Sergey
Re: [DISCUSS] KIP-271 Add NetworkClient redirector
Posted by Sergey Korytnikov <se...@gmail.com>.
Dear devs,
It's been a while since I created KIP-271 and received just a little
feedback. I'm going to start the voting process in a couple of days as I
feel responsible for the KIP created. Community needs to decide whether
the change is viable or not, especially, since kafka 1.0 when the change
has been originally proposed, so that the change has to be retested with
trunk head (> 2.1).
As part of the update from my side, I've added advertised.listeners
proposal as rejected alternative to KIP wiki page.
I still believe Kafka Client should have the flexibility to override the
routing rules in cases when the deployment environment with multiple
Kafka broker instances does not allow to fine tune each replica.
Thanks,
Sergey
On 3/17/2018 9:12 AM, Sergey Korytnikov wrote:
> Hi, Rajini,
>
> Thank you! This is really great advice, I will review that and test it
> out in our scenarios.
> Do you know any limitations when advertised.listeners may not work?
>
> Having the option to redirecting broker calls on client side is still
> valuable for kafka server setup that cannot have advertised.listeners
> configured for any reason.
>
> Thanks,
> Sergey
>
>
> On Fri, Mar 16, 2018 at 3:41 PM, Rajini Sivaram
> <rajinisivaram@gmail.com <ma...@gmail.com>> wrote:
>
> Hi Sergey,
>
> Thanks for the KIP. Is there a reason why brokers cannot be
> configured with
> `advertised.listeners` that use hostnames (or addresses) that
> clients can
> connect to, instead of performing a redirection?
>
> Regards,
>
> Rajini
>
> On Fri, Mar 16, 2018 at 1:24 PM, Sergey Korytnikov <
> sergey.korytnikov@gmail.com <ma...@gmail.com>>
> wrote:
>
> > Hi devs,
> >
> > I have created new KIP-271 for discussion:
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> <https://cwiki.apache.org/confluence/display/KAFKA/KIP->
> > 271%3A+Add+NetworkClient+redirector
> > Kafka JIRA issue:
> https://issues.apache.org/jira/browse/KAFKA-6669
> <https://issues.apache.org/jira/browse/KAFKA-6669>
> >
> > The fix has been implemented and tested with version 1.0 and
> trunk snapshot
> > of clients jar.
> > Changes are available here:
> > https://github.com/ssbiox/kafka/tree/trunk-redirector
> <https://github.com/ssbiox/kafka/tree/trunk-redirector>
> > Diff of changes:
> >
> https://github.com/apache/kafka/compare/trunk...ssbiox:trunk-redirector
> <https://github.com/apache/kafka/compare/trunk...ssbiox:trunk-redirector>
> >
> > Thank you,
> > Sergey
> >
>
>
Re: [DISCUSS] KIP-271 Add NetworkClient redirector
Posted by Sergey Korytnikov <se...@gmail.com>.
Hi, Rajini,
Thank you! This is really great advice, I will review that and test it out
in our scenarios.
Do you know any limitations when advertised.listeners may not work?
Having the option to redirecting broker calls on client side is still
valuable for kafka server setup that cannot have advertised.listeners
configured for any reason.
Thanks,
Sergey
On Fri, Mar 16, 2018 at 3:41 PM, Rajini Sivaram <ra...@gmail.com>
wrote:
> Hi Sergey,
>
> Thanks for the KIP. Is there a reason why brokers cannot be configured with
> `advertised.listeners` that use hostnames (or addresses) that clients can
> connect to, instead of performing a redirection?
>
> Regards,
>
> Rajini
>
> On Fri, Mar 16, 2018 at 1:24 PM, Sergey Korytnikov <
> sergey.korytnikov@gmail.com> wrote:
>
> > Hi devs,
> >
> > I have created new KIP-271 for discussion:
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 271%3A+Add+NetworkClient+redirector
> > Kafka JIRA issue: https://issues.apache.org/jira/browse/KAFKA-6669
> >
> > The fix has been implemented and tested with version 1.0 and trunk
> snapshot
> > of clients jar.
> > Changes are available here:
> > https://github.com/ssbiox/kafka/tree/trunk-redirector
> > Diff of changes:
> > https://github.com/apache/kafka/compare/trunk...ssbiox:trunk-redirector
> >
> > Thank you,
> > Sergey
> >
>
Re: [DISCUSS] KIP-271 Add NetworkClient redirector
Posted by Rajini Sivaram <ra...@gmail.com>.
Hi Sergey,
Thanks for the KIP. Is there a reason why brokers cannot be configured with
`advertised.listeners` that use hostnames (or addresses) that clients can
connect to, instead of performing a redirection?
Regards,
Rajini
On Fri, Mar 16, 2018 at 1:24 PM, Sergey Korytnikov <
sergey.korytnikov@gmail.com> wrote:
> Hi devs,
>
> I have created new KIP-271 for discussion:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 271%3A+Add+NetworkClient+redirector
> Kafka JIRA issue: https://issues.apache.org/jira/browse/KAFKA-6669
>
> The fix has been implemented and tested with version 1.0 and trunk snapshot
> of clients jar.
> Changes are available here:
> https://github.com/ssbiox/kafka/tree/trunk-redirector
> Diff of changes:
> https://github.com/apache/kafka/compare/trunk...ssbiox:trunk-redirector
>
> Thank you,
> Sergey
>