You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Chia-Ping Tsai <ch...@gmail.com> on 2025/11/29 20:02:11 UTC
Is the console output considered a public interface?
hi all,
I opened KAFKA-19941 to improve the feature tool output, but now we face a
compatibility question: Does our definition of a public interface, which
includes "monitoring" and "command line tools" (per the KIP page), cover
the output format?
any feedback?
Best,
Chia-Ping
Re: Is the console output considered a public interface?
Posted by Kirk True <ki...@kirktrue.pro>.
Hi all,
It would be good to come right and say that the output is not considered an "interface" if that's what's decided. Some of the command line tools produce structured output that suggest they could be relied on to scrape for various purposes.
Also, there are some system tests that rely on the output of command line tools, so we should encourage developers to run relevant system tests when the output format is changed. (Either that or change the tests.)
Thanks,
Kirk
On Mon, Dec 1, 2025, at 1:40 PM, Matthias J. Sax wrote:
> I also agree, that the output is not part of the public interface, for
> all the reason stated.
>
> When we refer to monitoring, my read is that it's about JMX metrics,
> which we cannot just add/remove or change.
>
> And when we refer to command lines tools, it's about the existence of a
> tool (we cannot just remove it), and it's input (ie, command line
> parameters), to not break any scripts / automation which is
> using/invoking the cli tools.
>
>
> -Matthias
>
> On 11/30/25 10:02 PM, jian fu wrote:
> > Hi:
> >
> > I feel that it makes more sense for the output not to be considered a
> > public interface. If we treat it as one, it implicitly suggests that the
> > output is a very stable dependency. Users might even build logic around
> > it—for example, counting how many lines contain certain keywords—which
> > would further restrict what we can do in the future. In reality, it’s just
> > output, and it’s something that can even be controlled by log4j to
> > determine whether it is displayed at all.
> >
> > Regards
> > Jian
> >
> > Andrew Schofield <an...@live.com> 于2025年12月1日周一 07:06写道：
> >
> >> Hi Chia-Ping,
> >> I’m going to jump in and get the discussion started.
> >>
> >> My view is that the output of the console tools is NOT a formal public
> >> interface. If it was, the KIPs would have to be totally explicit in the
> >> specification of the output formats, down to the spacing, column widths in
> >> tables, blank lines and so on. We would also need a major release to make
> >> even minor formatting changes unless they were protected by an option.
> >>
> >> I think the console tools are wrappers around the admin API, and that’s
> >> the public interface. If someone wants to build automation around Kafka, it
> >> would be much better to use the admin API rather than the console tools.
> >> For interfaces which are intended to have a human using them directly, I
> >> think we should be prepared to improve them in ways that a human will
> >> instantly understand but which might confuse a piece of unsophisticated
> >> software capturing and parsing the output.
> >>
> >> In a similar vein, would the log lines produced by a broker be considered
> >> a public interface? People will certainly be scraping those to understand
> >> broker information. I’d say that the log lines should also not be
> >> considered a formal public interface. I don’t think we want people relying
> >> on specific strings in the log lines, or the order in which they are
> >> emitted.
> >>
> >> Just my 2 cents,
> >> Andrew
> >>
> >>
> >> On 29 Nov 2025, at 20:02, Chia-Ping Tsai <ch...@gmail.com> wrote:
> >>
> >> hi all,
> >>
> >> I opened KAFKA-19941 to improve the feature tool output, but now we face a
> >> compatibility question: Does our definition of a public interface, which
> >> includes "monitoring" and "command line tools" (per the KIP page), cover
> >> the output format?
> >>
> >> any feedback?
> >>
> >> Best,
> >> Chia-Ping
> >>
> >>
> >
>
>
Re: Is the console output considered a public interface?
Posted by "Matthias J. Sax" <mj...@apache.org>.
I also agree, that the output is not part of the public interface, for
all the reason stated.
When we refer to monitoring, my read is that it's about JMX metrics,
which we cannot just add/remove or change.
And when we refer to command lines tools, it's about the existence of a
tool (we cannot just remove it), and it's input (ie, command line
parameters), to not break any scripts / automation which is
using/invoking the cli tools.
-Matthias
On 11/30/25 10:02 PM, jian fu wrote:
> Hi:
>
> I feel that it makes more sense for the output not to be considered a
> public interface. If we treat it as one, it implicitly suggests that the
> output is a very stable dependency. Users might even build logic around
> it—for example, counting how many lines contain certain keywords—which
> would further restrict what we can do in the future. In reality, it’s just
> output, and it’s something that can even be controlled by log4j to
> determine whether it is displayed at all.
>
> Regards
> Jian
>
> Andrew Schofield <an...@live.com> 于2025年12月1日周一 07:06写道：
>
>> Hi Chia-Ping,
>> I’m going to jump in and get the discussion started.
>>
>> My view is that the output of the console tools is NOT a formal public
>> interface. If it was, the KIPs would have to be totally explicit in the
>> specification of the output formats, down to the spacing, column widths in
>> tables, blank lines and so on. We would also need a major release to make
>> even minor formatting changes unless they were protected by an option.
>>
>> I think the console tools are wrappers around the admin API, and that’s
>> the public interface. If someone wants to build automation around Kafka, it
>> would be much better to use the admin API rather than the console tools.
>> For interfaces which are intended to have a human using them directly, I
>> think we should be prepared to improve them in ways that a human will
>> instantly understand but which might confuse a piece of unsophisticated
>> software capturing and parsing the output.
>>
>> In a similar vein, would the log lines produced by a broker be considered
>> a public interface? People will certainly be scraping those to understand
>> broker information. I’d say that the log lines should also not be
>> considered a formal public interface. I don’t think we want people relying
>> on specific strings in the log lines, or the order in which they are
>> emitted.
>>
>> Just my 2 cents,
>> Andrew
>>
>>
>> On 29 Nov 2025, at 20:02, Chia-Ping Tsai <ch...@gmail.com> wrote:
>>
>> hi all,
>>
>> I opened KAFKA-19941 to improve the feature tool output, but now we face a
>> compatibility question: Does our definition of a public interface, which
>> includes "monitoring" and "command line tools" (per the KIP page), cover
>> the output format?
>>
>> any feedback?
>>
>> Best,
>> Chia-Ping
>>
>>
>
Re: Is the console output considered a public interface?
Posted by jian fu <fu...@gmail.com>.
Hi:
I feel that it makes more sense for the output not to be considered a
public interface. If we treat it as one, it implicitly suggests that the
output is a very stable dependency. Users might even build logic around
it—for example, counting how many lines contain certain keywords—which
would further restrict what we can do in the future. In reality, it’s just
output, and it’s something that can even be controlled by log4j to
determine whether it is displayed at all.
Regards
Jian
Andrew Schofield <an...@live.com> 于2025年12月1日周一 07:06写道：
> Hi Chia-Ping,
> I’m going to jump in and get the discussion started.
>
> My view is that the output of the console tools is NOT a formal public
> interface. If it was, the KIPs would have to be totally explicit in the
> specification of the output formats, down to the spacing, column widths in
> tables, blank lines and so on. We would also need a major release to make
> even minor formatting changes unless they were protected by an option.
>
> I think the console tools are wrappers around the admin API, and that’s
> the public interface. If someone wants to build automation around Kafka, it
> would be much better to use the admin API rather than the console tools.
> For interfaces which are intended to have a human using them directly, I
> think we should be prepared to improve them in ways that a human will
> instantly understand but which might confuse a piece of unsophisticated
> software capturing and parsing the output.
>
> In a similar vein, would the log lines produced by a broker be considered
> a public interface? People will certainly be scraping those to understand
> broker information. I’d say that the log lines should also not be
> considered a formal public interface. I don’t think we want people relying
> on specific strings in the log lines, or the order in which they are
> emitted.
>
> Just my 2 cents,
> Andrew
>
>
> On 29 Nov 2025, at 20:02, Chia-Ping Tsai <ch...@gmail.com> wrote:
>
> hi all,
>
> I opened KAFKA-19941 to improve the feature tool output, but now we face a
> compatibility question: Does our definition of a public interface, which
> includes "monitoring" and "command line tools" (per the KIP page), cover
> the output format?
>
> any feedback?
>
> Best,
> Chia-Ping
>
>
Re: Is the console output considered a public interface?
Posted by Andrew Schofield <an...@live.com>.
Hi Chia-Ping,
I’m going to jump in and get the discussion started.
My view is that the output of the console tools is NOT a formal public interface. If it was, the KIPs would have to be totally explicit in the specification of the output formats, down to the spacing, column widths in tables, blank lines and so on. We would also need a major release to make even minor formatting changes unless they were protected by an option.
I think the console tools are wrappers around the admin API, and that’s the public interface. If someone wants to build automation around Kafka, it would be much better to use the admin API rather than the console tools. For interfaces which are intended to have a human using them directly, I think we should be prepared to improve them in ways that a human will instantly understand but which might confuse a piece of unsophisticated software capturing and parsing the output.
In a similar vein, would the log lines produced by a broker be considered a public interface? People will certainly be scraping those to understand broker information. I’d say that the log lines should also not be considered a formal public interface. I don’t think we want people relying on specific strings in the log lines, or the order in which they are emitted.
Just my 2 cents,
Andrew
On 29 Nov 2025, at 20:02, Chia-Ping Tsai <ch...@gmail.com> wrote:
hi all,
I opened KAFKA-19941 to improve the feature tool output, but now we face a
compatibility question: Does our definition of a public interface, which
includes "monitoring" and "command line tools" (per the KIP page), cover
the output format?
any feedback?
Best,
Chia-Ping
Re: Is the console output considered a public interface?
Posted by Chia-Ping Tsai <ch...@apache.org>.
Thanks for all feedback. I have added following statement to KIP page
> Neither the console tool output nor the log content is considered a formal public interface (see discussion https://lists.apache.org/thread/jwcn88o4f4c19x1onymxmkddk3dcsckf)
On 2025/11/29 20:02:11 Chia-Ping Tsai wrote:
> hi all,
>
> I opened KAFKA-19941 to improve the feature tool output, but now we face a
> compatibility question: Does our definition of a public interface, which
> includes "monitoring" and "command line tools" (per the KIP page), cover
> the output format?
>
> any feedback?
>
> Best,
> Chia-Ping
>