The way for the tools and the Admin Client API usage

[Paolo Patierno <pp...@live.com>]

Hi devs and committers,

I'd like to raise a point about Kafka tools and Admin Client API in order to have a sort of agreement on the path we should follow for having consistency in the project.

We all know that the bigger part of tools is in Scala with only few of them in Java. At same time more of them are still using Zookeeper and few of them are using Admin Client API (even because it's quite new and evolving release by release).


My initial understanding was that the community (at least the committers) would like to have tools migrated in Java (something like already happened for clients) and using the new Admin Client API.

This was the path I was trying to follow with TopicCommand tool then I noticed that ...


A new delete records tool was added (the PR was merged) and it was written in Scala, adding the new protocol commands for doing that to the "legacy" Admin client (for this I opened the KIP-204<https://cwiki.apache.org/confluence/display/KAFKA/KIP-204+%3A+Adding+records+deletion+operation+to+the+new+Admin+Client+API> for having it in the new Admin Client API). So, to summarize, new tool ... no Java ... no new Admin Client API usage.


What I see is that there is a real "mix" : Scala tools using "legacy" Admin Client, Scala tools using new Admin Client API, few Java tools just using consumer/producer clients, ...


In the past, during this discussion<https://www.mail-archive.com/dev@kafka.apache.org/msg78014.html>, the final idea was having the bash script layer using different tools (Scala or Java) based on Zookeeper parameter usage.


I'd like to know the opinion about the final objective and the path to follow around tools. Maybe the final objective should be having all tools in Java using Admin Client API but maybe during this path having a smooth migration just using new Admin Client API in the current Scala tools first (removing Zookeeper calls) could be better. Maybe for committers, in order to review and merge PRs, little ones are better ...


What do you think about this ?


Thanks,


Paolo Patierno
Senior Software Engineer (IoT) @ Red Hat
Microsoft MVP on Azure & IoT
Microsoft Azure Advisor

Twitter : @ppatierno<http://twitter.com/ppatierno>
Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno>
Blog : DevExperience<http://paolopatierno.wordpress.com/>

Re: The way for the tools and the Admin Client API usage

[Ismael Juma <is...@juma.me.uk>]

Hi Paolo,

The delete records KIP was voted and implemented before the AdminClient
KIP. It couldn't use something that didn't exist. :)

Regarding which part to take, I would say it depends on the contributor.
Both moving the tool to `tools` and removing the ZK dependency are
valuable. If someone is willing to do both, that's great. In cases where
removing the ZK dependency is a small change when compared to moving the
tool to `tools`, then we would accept contributions that do the former as
well. If it's a large change either way, we'd prefer to do both at the same
time.

Ismael

On Tue, Sep 26, 2017 at 10:34 AM, Paolo Patierno <pp...@live.com> wrote:

> Hi devs and committers,
>
> I'd like to raise a point about Kafka tools and Admin Client API in order
> to have a sort of agreement on the path we should follow for having
> consistency in the project.
>
> We all know that the bigger part of tools is in Scala with only few of
> them in Java. At same time more of them are still using Zookeeper and few
> of them are using Admin Client API (even because it's quite new and
> evolving release by release).
>
>
> My initial understanding was that the community (at least the committers)
> would like to have tools migrated in Java (something like already happened
> for clients) and using the new Admin Client API.
>
> This was the path I was trying to follow with TopicCommand tool then I
> noticed that ...
>
>
> A new delete records tool was added (the PR was merged) and it was written
> in Scala, adding the new protocol commands for doing that to the "legacy"
> Admin client (for this I opened the KIP-204<https://cwiki.apache.
> org/confluence/display/KAFKA/KIP-204+%3A+Adding+records+
> deletion+operation+to+the+new+Admin+Client+API> for having it in the new
> Admin Client API). So, to summarize, new tool ... no Java ... no new Admin
> Client API usage.
>
>
> What I see is that there is a real "mix" : Scala tools using "legacy"
> Admin Client, Scala tools using new Admin Client API, few Java tools just
> using consumer/producer clients, ...
>
>
> In the past, during this discussion<https://www.mail-
> archive.com/dev@kafka.apache.org/msg78014.html>, the final idea was
> having the bash script layer using different tools (Scala or Java) based on
> Zookeeper parameter usage.
>
>
> I'd like to know the opinion about the final objective and the path to
> follow around tools. Maybe the final objective should be having all tools
> in Java using Admin Client API but maybe during this path having a smooth
> migration just using new Admin Client API in the current Scala tools first
> (removing Zookeeper calls) could be better. Maybe for committers, in order
> to review and merge PRs, little ones are better ...
>
>
> What do you think about this ?
>
>
> Thanks,
>
>
> Paolo Patierno
> Senior Software Engineer (IoT) @ Red Hat
> Microsoft MVP on Azure & IoT
> Microsoft Azure Advisor
>
> Twitter : @ppatierno<http://twitter.com/ppatierno>
> Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno>
> Blog : DevExperience<http://paolopatierno.wordpress.com/>
>