You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Ashwin via dev <de...@kafka.apache.org> on 2026/01/05 04:21:10 UTC
[DISCUSS] KIP-1265: Public API needs to be explicitly declared
Hello folks,
Reviving this old thread
https://lists.apache.org/thread/ly5ddkobr1wc07gvhwc1p0jg94qg8cxc to discuss
KIP-1265.
Apache Kafka lacks a concrete, centralized definition of what constitutes a
public API. The most relevant information currently available is found here:
Kafka Improvement Proposals#Whatisconsidereda%22majorchange%22thatneedsaKIP
<https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaKIP>
Without formal definition or guardrails, there is a risk that builders may
inadvertently import internal classes leading to possible build breakages
when they compile against a newer Kafka version
Please let me know your thoughts for
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1265%3A+Public+API+needs+to+be+explicitly+declared
Cheers,
Ashwin
Re: [DISCUSS] KIP-1265: Public API needs to be explicitly declared
Posted by Manikumar <ma...@gmail.com>.
Hi Ashwin,
Thanks for the update. The KIP changes look good to me.
We should also make sure to update the release scripts/documentation
to publish the new artifacts, along with usage documentation.
Thanks,
Manikumar
On Wed, May 13, 2026 at 11:44 AM Ashwin <ap...@confluent.io> wrote:
>
> Hi Manikumar,
>
> Thank you so much for reviewing the KIP.
>
> 1. I get your point regarding InterfaceAudience (we had earlier discussed this in the thread , there were no strong opinions for or against it). I have updated the KIP with this new annotation. I have not added LimitedPrivate - as I don't think we can use it to enforce checks for Plugin developers (I have mentioned this in the KIP)
> 2. That is a great point ! I have updated the KIP with this approach.
> 3. Fair point - updated in the KIP
> 4. Agreed.
>
> Cheers,
> Ashwin
>
>
> On Tue, May 12, 2026 at 4:24 PM Manikumar <ma...@gmail.com> wrote:
>>
>> Hi Ashwin,
>>
>> Thanks for working on this. A few questions/suggestions:
>>
>> 1. Relationship with the existing InterfaceStability annotation. We
>> already have @InterfaceStability.{Stable,Evolving,Unstable} applied
>> across the codebase,
>> and it looks like Kafka's InterfaceStability was originally borrowed
>> from Hadoop's @InterfaceStability:
>> https://urldefense.com/v3/__https://hadoop.apache.org/docs/r2.7.1/api/org/apache/hadoop/classification/InterfaceStability.html__;!!Ayb5sqE7!rDuqH6O-GV8sRTDtC9qrhH1DSsZ7-VSm479gcYux9CAW_qzLfPH1m8rCD1s44MZysaKCJJ18aySHVzYbNGtcTwW3k68T$
>>
>> In Hadoop, InterfaceStability is paired with @InterfaceAudience
>> (Public / LimitedPrivate / Private), which is a richer model than a
>> single @PublicApi marker:
>> https://urldefense.com/v3/__https://hadoop.apache.org/docs/r2.7.1/api/org/apache/hadoop/classification/InterfaceAudience.html__;!!Ayb5sqE7!rDuqH6O-GV8sRTDtC9qrhH1DSsZ7-VSm479gcYux9CAW_qzLfPH1m8rCD1s44MZysaKCJJ18aySHVzYbNGtcTzz1tYf2$
>>
>> Could we consider borrowing @InterfaceAudience as well, instead of
>> introducing a separate @PublicApi marker? That would let us:
>> - Cleanly distinguish "public to end users" vs "public to
>> connector/plugin developers only" vs "internal"
>> - Keep the audience and stability dimensions orthogonal, matching
>> what Hadoop has proven works
>> - Avoid the ambiguity of how @PublicApi interacts with
>> @InterfaceStability.Evolving
>>
>> If we go with a new @PublicApi instead, could the KIP explicitly say
>> how it relates to @InterfaceStability (replace it, coexist with it,
>> etc.)?
>>
>> 2. External check robustness. The external-usage check regex-scans
>> .java source files for imports. This misses Scala/Kotlin consumers and
>> fully-qualified usages. Have you considered an ASM-based bytecode scan
>> instead? That would catch all JVM-language consumers uniformly.
>>
>> 3. Suppression mechanism. Is there a way to mark a known/intentional
>> violation? Without one, the checker can't be turned on in projects
>> that already have legitimate exposures.
>>
>> 4. I would suggest making the annotation @Documented so it surfaces
>> in javadoc, and adding a since attribute.
>>
>> Thanks,
>> Manikumar
Re: [DISCUSS] KIP-1265: Public API needs to be explicitly declared
Posted by Ashwin via dev <de...@kafka.apache.org>.
Hi Manikumar,
Thank you so much for reviewing the KIP.
1. I get your point regarding InterfaceAudience (we had earlier discussed
this in the thread , there were no strong opinions for or against it). I
have updated the KIP with this new annotation. I have not added
LimitedPrivate - as I don't think we can use it to enforce checks for
Plugin developers (I have mentioned this in the KIP)
2. That is a great point ! I have updated the KIP with this approach.
3. Fair point - updated in the KIP
4. Agreed.
Cheers,
Ashwin
On Tue, May 12, 2026 at 4:24 PM Manikumar <ma...@gmail.com> wrote:
> Hi Ashwin,
>
> Thanks for working on this. A few questions/suggestions:
>
> 1. Relationship with the existing InterfaceStability annotation. We
> already have @InterfaceStability.{Stable,Evolving,Unstable} applied
> across the codebase,
> and it looks like Kafka's InterfaceStability was originally borrowed
> from Hadoop's @InterfaceStability:
>
> https://urldefense.com/v3/__https://hadoop.apache.org/docs/r2.7.1/api/org/apache/hadoop/classification/InterfaceStability.html__;!!Ayb5sqE7!rDuqH6O-GV8sRTDtC9qrhH1DSsZ7-VSm479gcYux9CAW_qzLfPH1m8rCD1s44MZysaKCJJ18aySHVzYbNGtcTwW3k68T$
>
> In Hadoop, InterfaceStability is paired with @InterfaceAudience
> (Public / LimitedPrivate / Private), which is a richer model than a
> single @PublicApi marker:
>
> https://urldefense.com/v3/__https://hadoop.apache.org/docs/r2.7.1/api/org/apache/hadoop/classification/InterfaceAudience.html__;!!Ayb5sqE7!rDuqH6O-GV8sRTDtC9qrhH1DSsZ7-VSm479gcYux9CAW_qzLfPH1m8rCD1s44MZysaKCJJ18aySHVzYbNGtcTzz1tYf2$
>
> Could we consider borrowing @InterfaceAudience as well, instead of
> introducing a separate @PublicApi marker? That would let us:
> - Cleanly distinguish "public to end users" vs "public to
> connector/plugin developers only" vs "internal"
> - Keep the audience and stability dimensions orthogonal, matching
> what Hadoop has proven works
> - Avoid the ambiguity of how @PublicApi interacts with
> @InterfaceStability.Evolving
>
> If we go with a new @PublicApi instead, could the KIP explicitly say
> how it relates to @InterfaceStability (replace it, coexist with it,
> etc.)?
>
> 2. External check robustness. The external-usage check regex-scans
> .java source files for imports. This misses Scala/Kotlin consumers and
> fully-qualified usages. Have you considered an ASM-based bytecode scan
> instead? That would catch all JVM-language consumers uniformly.
>
> 3. Suppression mechanism. Is there a way to mark a known/intentional
> violation? Without one, the checker can't be turned on in projects
> that already have legitimate exposures.
>
> 4. I would suggest making the annotation @Documented so it surfaces
> in javadoc, and adding a since attribute.
>
> Thanks,
> Manikumar
>
Re: [DISCUSS] KIP-1265: Public API needs to be explicitly declared
Posted by Manikumar <ma...@gmail.com>.
Hi Ashwin,
Thanks for working on this. A few questions/suggestions:
1. Relationship with the existing InterfaceStability annotation. We
already have @InterfaceStability.{Stable,Evolving,Unstable} applied
across the codebase,
and it looks like Kafka's InterfaceStability was originally borrowed
from Hadoop's @InterfaceStability:
https://hadoop.apache.org/docs/r2.7.1/api/org/apache/hadoop/classification/InterfaceStability.html
In Hadoop, InterfaceStability is paired with @InterfaceAudience
(Public / LimitedPrivate / Private), which is a richer model than a
single @PublicApi marker:
https://hadoop.apache.org/docs/r2.7.1/api/org/apache/hadoop/classification/InterfaceAudience.html
Could we consider borrowing @InterfaceAudience as well, instead of
introducing a separate @PublicApi marker? That would let us:
- Cleanly distinguish "public to end users" vs "public to
connector/plugin developers only" vs "internal"
- Keep the audience and stability dimensions orthogonal, matching
what Hadoop has proven works
- Avoid the ambiguity of how @PublicApi interacts with
@InterfaceStability.Evolving
If we go with a new @PublicApi instead, could the KIP explicitly say
how it relates to @InterfaceStability (replace it, coexist with it,
etc.)?
2. External check robustness. The external-usage check regex-scans
.java source files for imports. This misses Scala/Kotlin consumers and
fully-qualified usages. Have you considered an ASM-based bytecode scan
instead? That would catch all JVM-language consumers uniformly.
3. Suppression mechanism. Is there a way to mark a known/intentional
violation? Without one, the checker can't be turned on in projects
that already have legitimate exposures.
4. I would suggest making the annotation @Documented so it surfaces
in javadoc, and adding a since attribute.
Thanks,
Manikumar
Re: [DISCUSS] KIP-1265: Public API needs to be explicitly declared
Posted by Ashwin via dev <de...@kafka.apache.org>.
Hi everyone,
Please let me know if there is any further feedback or ways I can improve
this KIP. As I haven't been able to secure enough votes in the voting
thread yet, I would appreciate any guidance on the next steps to move this
proposal forward.
Thanks,
Ashwin
On Thu, Feb 12, 2026 at 9:19 PM Ashwin <ap...@confluent.io> wrote:
> Hello Folks,
>
> I am reviving this thread since I did not receive any votes in the voting
> thread - https://lists.apache.org/thread/9wwrx3m7s5k05hxkqd4ygdfpv0n50f0r
> .
> I request everyone to please share your feedback on how I can improve the
> KIP, so that I can get more engagement in the voting thread.
> I believe that I have incorporated all the feedback which I have received
> so far.
>
> Cheers,
> Ashwin
>
> On Fri, Jan 23, 2026 at 8:46 AM Ashwin <ap...@confluent.io> wrote:
>
>> Hello folks,
>>
>> I have created a draft PR to demonstrate the changes involved:
>> https://github.com/apache/kafka/pull/21337/files .
>>
>> In this PR, I have annotated the KafkaProducer class with @PublicApi. Any
>> "un-annotated" classes present in the Javadoc jar will be reported,
>> causing the final docsJar step to fail. This public API checker will
>> also check the public methods, args and exception types - to avoid
>> unintentional exposure of internal classes.
>>
>> For the plugin developer's reference, I have created a test repository
>> using the new Gradle and Maven checkers:
>>
>> - Gradle:
>> https://github.com/ashwinpankaj/kip1265test/blob/58d4f88f54d8324fb76656f24ad4334e483cdd55/gradle-test/my-simple-app/build.gradle#L60-L64
>>
>> - Maven:
>> https://github.com/ashwinpankaj/kip1265test/blob/58d4f88f54d8324fb76656f24ad4334e483cdd55/maven-test/my-simple-app/pom.xml#L44-L71
>>
>>
>> Both builds in this repository pass for KafkaProducer but fail for other
>> imports from org.apache.kafka.*.
>>
>> Please let me know your thoughts, as I plan to start the vote for this
>> KIP next week.
>>
>> Thanks,
>> Ashwin
>>
>> On Tue, Jan 20, 2026 at 6:52 PM Ashwin <ap...@confluent.io> wrote:
>>
>>> Thanks for providing feedback Stan ! Sorry I couldn't reply earlier .
>>>
>>> I too was having second thoughts about Plugin development support which
>>> I proposed earlier . I now feel that we should publish Gradle and Maven
>>> plugins from Apache Kafka. Developers can then use these plugins in their
>>> build config.
>>>
>>> I have published the proposal at
>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=406618601#KIP1265:MechanismforautomaticdetectionofinternalAPIusage-GuardrailsforPlugindevelopers
>>> .
>>>
>>> Please have a look and let me know what you think.
>>>
>>> Cheers,
>>> Ashwin
>>>
>>> On Wed, Jan 14, 2026 at 1:53 AM Stanislav Kozlovski <
>>> stanislavkozlovski@apache.org> wrote:
>>>
>>>> Thanks for working on this! It's much needed, I remember many times
>>>> where I'd catch similar misses in PR reviews in the last minute.
>>>>
>>>> As for helping Plugin devs, how do we precisely plan to "provide a
>>>> sample configuration" and "publish a sample APILyzer configuration". Do we
>>>> expect people to know where to find these things from the docs and copy
>>>> them to their own repos?
>>>>
>>>> Can you think of any ways to warn Kafka users, i.e people who import
>>>> `org.apache.kafka.Something`, of the fact they're depending on an unstable
>>>> API too?
>>>>
>>>> Best,
>>>> Stan
>>>>
>>>> On 2026/01/08 07:21:23 Ashwin via dev wrote:
>>>> > Thanks for your feedback Ismael ! I was not aware of the
>>>> InterfaceStability
>>>> > annotation !
>>>> > I have updated the KIP based on your suggestions
>>>> >
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1265%3A+Mechanism+for+automatic+detection+of+internal+API+usage
>>>> > and hope that all your points have been added to it.
>>>> >
>>>> > Team - Please do let me know your thoughts.
>>>> > Thanks,
>>>> > Ashwin
>>>> >
>>>> > On Tue, Jan 6, 2026 at 6:10 AM Ismael Juma <me...@ismaeljuma.com> wrote:
>>>> >
>>>> > > Hi Ashwin,
>>>> > >
>>>> > > Improving how we specify public API and providing a mechanism for
>>>> > > automatic detection of internal api usage would be great. That
>>>> said, it's
>>>> > > incorrect that there is no mechanism for that today. We should
>>>> update the
>>>> > > KIP to specify the current situation correctly and the problems
>>>> with it.
>>>> > >
>>>> > > A summary of items to consider for the KIP:
>>>> > > 1. As a user, the way to know the public API is to consult the
>>>> project's
>>>> > > published javadoc: this is a "concrete, centralized definition of
>>>> what
>>>> > > constitutes a public API".
>>>> > > 2. As a Kafka developer, you specify it via a block like this:
>>>> > > https://github.com/apache/kafka/blob/trunk/build.gradle#L2026
>>>> > > 3. There is no tooling that automatically detects if a public method
>>>> > > exposes an internal class (there was a recent KIP that was
>>>> introduced
>>>> > > because we accidentally exposed an internal class)
>>>> > > 4. There is no simple mechanism for users of Kafka to configure
>>>> their
>>>> > > build system so that usage of Kafka internal apis are flagged.
>>>> > > 5. There are already annotations to specify whether a class or
>>>> method is
>>>> > > stable, evolving or unstable:
>>>> > >
>>>> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/annotation/InterfaceStability.java
>>>> > >
>>>> > > Thanks,
>>>> > > Ismael
>>>> > >
>>>> > > On Sun, Jan 4, 2026 at 8:21 PM Ashwin via dev <dev@kafka.apache.org
>>>> >
>>>> > > wrote:
>>>> > >
>>>> > >> Hello folks,
>>>> > >>
>>>> > >> Reviving this old thread
>>>> > >> https://lists.apache.org/thread/ly5ddkobr1wc07gvhwc1p0jg94qg8cxc
>>>> to
>>>> > >> discuss
>>>> > >> KIP-1265.
>>>> > >>
>>>> > >> Apache Kafka lacks a concrete, centralized definition of what
>>>> constitutes
>>>> > >> a
>>>> > >> public API. The most relevant information currently available is
>>>> found
>>>> > >> here:
>>>> > >>
>>>> > >> Kafka Improvement
>>>> > >> Proposals#Whatisconsidereda%22majorchange%22thatneedsaKIP
>>>> > >> <
>>>> > >>
>>>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaKIP
>>>> > >> >
>>>> > >>
>>>> > >> Without formal definition or guardrails, there is a risk that
>>>> builders may
>>>> > >> inadvertently import internal classes leading to possible build
>>>> breakages
>>>> > >> when they compile against a newer Kafka version
>>>> > >>
>>>> > >>
>>>> > >> Please let me know your thoughts for
>>>> > >>
>>>> > >>
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1265%3A+Public+API+needs+to+be+explicitly+declared
>>>> > >>
>>>> > >> Cheers,
>>>> > >> Ashwin
>>>> > >>
>>>> > >
>>>> >
>>>>
>>>
Re: [DISCUSS] KIP-1265: Public API needs to be explicitly declared
Posted by Ashwin via dev <de...@kafka.apache.org>.
Hello Folks,
I am reviving this thread since I did not receive any votes in the voting
thread - https://lists.apache.org/thread/9wwrx3m7s5k05hxkqd4ygdfpv0n50f0r .
I request everyone to please share your feedback on how I can improve the
KIP, so that I can get more engagement in the voting thread.
I believe that I have incorporated all the feedback which I have received
so far.
Cheers,
Ashwin
On Fri, Jan 23, 2026 at 8:46 AM Ashwin <ap...@confluent.io> wrote:
> Hello folks,
>
> I have created a draft PR to demonstrate the changes involved:
> https://github.com/apache/kafka/pull/21337/files .
>
> In this PR, I have annotated the KafkaProducer class with @PublicApi. Any
> "un-annotated" classes present in the Javadoc jar will be reported,
> causing the final docsJar step to fail. This public API checker will also
> check the public methods, args and exception types - to avoid unintentional
> exposure of internal classes.
>
> For the plugin developer's reference, I have created a test repository
> using the new Gradle and Maven checkers:
>
> - Gradle:
> https://github.com/ashwinpankaj/kip1265test/blob/58d4f88f54d8324fb76656f24ad4334e483cdd55/gradle-test/my-simple-app/build.gradle#L60-L64
>
> - Maven:
> https://github.com/ashwinpankaj/kip1265test/blob/58d4f88f54d8324fb76656f24ad4334e483cdd55/maven-test/my-simple-app/pom.xml#L44-L71
>
>
> Both builds in this repository pass for KafkaProducer but fail for other
> imports from org.apache.kafka.*.
>
> Please let me know your thoughts, as I plan to start the vote for this KIP
> next week.
>
> Thanks,
> Ashwin
>
> On Tue, Jan 20, 2026 at 6:52 PM Ashwin <ap...@confluent.io> wrote:
>
>> Thanks for providing feedback Stan ! Sorry I couldn't reply earlier .
>>
>> I too was having second thoughts about Plugin development support which I
>> proposed earlier . I now feel that we should publish Gradle and Maven
>> plugins from Apache Kafka. Developers can then use these plugins in their
>> build config.
>>
>> I have published the proposal at
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=406618601#KIP1265:MechanismforautomaticdetectionofinternalAPIusage-GuardrailsforPlugindevelopers
>> .
>>
>> Please have a look and let me know what you think.
>>
>> Cheers,
>> Ashwin
>>
>> On Wed, Jan 14, 2026 at 1:53 AM Stanislav Kozlovski <
>> stanislavkozlovski@apache.org> wrote:
>>
>>> Thanks for working on this! It's much needed, I remember many times
>>> where I'd catch similar misses in PR reviews in the last minute.
>>>
>>> As for helping Plugin devs, how do we precisely plan to "provide a
>>> sample configuration" and "publish a sample APILyzer configuration". Do we
>>> expect people to know where to find these things from the docs and copy
>>> them to their own repos?
>>>
>>> Can you think of any ways to warn Kafka users, i.e people who import
>>> `org.apache.kafka.Something`, of the fact they're depending on an unstable
>>> API too?
>>>
>>> Best,
>>> Stan
>>>
>>> On 2026/01/08 07:21:23 Ashwin via dev wrote:
>>> > Thanks for your feedback Ismael ! I was not aware of the
>>> InterfaceStability
>>> > annotation !
>>> > I have updated the KIP based on your suggestions
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1265%3A+Mechanism+for+automatic+detection+of+internal+API+usage
>>> > and hope that all your points have been added to it.
>>> >
>>> > Team - Please do let me know your thoughts.
>>> > Thanks,
>>> > Ashwin
>>> >
>>> > On Tue, Jan 6, 2026 at 6:10 AM Ismael Juma <me...@ismaeljuma.com> wrote:
>>> >
>>> > > Hi Ashwin,
>>> > >
>>> > > Improving how we specify public API and providing a mechanism for
>>> > > automatic detection of internal api usage would be great. That said,
>>> it's
>>> > > incorrect that there is no mechanism for that today. We should
>>> update the
>>> > > KIP to specify the current situation correctly and the problems with
>>> it.
>>> > >
>>> > > A summary of items to consider for the KIP:
>>> > > 1. As a user, the way to know the public API is to consult the
>>> project's
>>> > > published javadoc: this is a "concrete, centralized definition of
>>> what
>>> > > constitutes a public API".
>>> > > 2. As a Kafka developer, you specify it via a block like this:
>>> > > https://github.com/apache/kafka/blob/trunk/build.gradle#L2026
>>> > > 3. There is no tooling that automatically detects if a public method
>>> > > exposes an internal class (there was a recent KIP that was introduced
>>> > > because we accidentally exposed an internal class)
>>> > > 4. There is no simple mechanism for users of Kafka to configure their
>>> > > build system so that usage of Kafka internal apis are flagged.
>>> > > 5. There are already annotations to specify whether a class or
>>> method is
>>> > > stable, evolving or unstable:
>>> > >
>>> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/annotation/InterfaceStability.java
>>> > >
>>> > > Thanks,
>>> > > Ismael
>>> > >
>>> > > On Sun, Jan 4, 2026 at 8:21 PM Ashwin via dev <de...@kafka.apache.org>
>>> > > wrote:
>>> > >
>>> > >> Hello folks,
>>> > >>
>>> > >> Reviving this old thread
>>> > >> https://lists.apache.org/thread/ly5ddkobr1wc07gvhwc1p0jg94qg8cxc to
>>> > >> discuss
>>> > >> KIP-1265.
>>> > >>
>>> > >> Apache Kafka lacks a concrete, centralized definition of what
>>> constitutes
>>> > >> a
>>> > >> public API. The most relevant information currently available is
>>> found
>>> > >> here:
>>> > >>
>>> > >> Kafka Improvement
>>> > >> Proposals#Whatisconsidereda%22majorchange%22thatneedsaKIP
>>> > >> <
>>> > >>
>>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaKIP
>>> > >> >
>>> > >>
>>> > >> Without formal definition or guardrails, there is a risk that
>>> builders may
>>> > >> inadvertently import internal classes leading to possible build
>>> breakages
>>> > >> when they compile against a newer Kafka version
>>> > >>
>>> > >>
>>> > >> Please let me know your thoughts for
>>> > >>
>>> > >>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1265%3A+Public+API+needs+to+be+explicitly+declared
>>> > >>
>>> > >> Cheers,
>>> > >> Ashwin
>>> > >>
>>> > >
>>> >
>>>
>>
Re: [DISCUSS] KIP-1265: Public API needs to be explicitly declared
Posted by Ashwin via dev <de...@kafka.apache.org>.
Hello folks,
I have created a draft PR to demonstrate the changes involved:
https://github.com/apache/kafka/pull/21337/files .
In this PR, I have annotated the KafkaProducer class with @PublicApi. Any "
un-annotated" classes present in the Javadoc jar will be reported, causing
the final docsJar step to fail. This public API checker will also check the
public methods, args and exception types - to avoid unintentional exposure
of internal classes.
For the plugin developer's reference, I have created a test repository
using the new Gradle and Maven checkers:
- Gradle:
https://github.com/ashwinpankaj/kip1265test/blob/58d4f88f54d8324fb76656f24ad4334e483cdd55/gradle-test/my-simple-app/build.gradle#L60-L64
- Maven:
https://github.com/ashwinpankaj/kip1265test/blob/58d4f88f54d8324fb76656f24ad4334e483cdd55/maven-test/my-simple-app/pom.xml#L44-L71
Both builds in this repository pass for KafkaProducer but fail for other
imports from org.apache.kafka.*.
Please let me know your thoughts, as I plan to start the vote for this KIP
next week.
Thanks,
Ashwin
On Tue, Jan 20, 2026 at 6:52 PM Ashwin <ap...@confluent.io> wrote:
> Thanks for providing feedback Stan ! Sorry I couldn't reply earlier .
>
> I too was having second thoughts about Plugin development support which I
> proposed earlier . I now feel that we should publish Gradle and Maven
> plugins from Apache Kafka. Developers can then use these plugins in their
> build config.
>
> I have published the proposal at
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=406618601#KIP1265:MechanismforautomaticdetectionofinternalAPIusage-GuardrailsforPlugindevelopers
> .
>
> Please have a look and let me know what you think.
>
> Cheers,
> Ashwin
>
> On Wed, Jan 14, 2026 at 1:53 AM Stanislav Kozlovski <
> stanislavkozlovski@apache.org> wrote:
>
>> Thanks for working on this! It's much needed, I remember many times where
>> I'd catch similar misses in PR reviews in the last minute.
>>
>> As for helping Plugin devs, how do we precisely plan to "provide a sample
>> configuration" and "publish a sample APILyzer configuration". Do we expect
>> people to know where to find these things from the docs and copy them to
>> their own repos?
>>
>> Can you think of any ways to warn Kafka users, i.e people who import
>> `org.apache.kafka.Something`, of the fact they're depending on an unstable
>> API too?
>>
>> Best,
>> Stan
>>
>> On 2026/01/08 07:21:23 Ashwin via dev wrote:
>> > Thanks for your feedback Ismael ! I was not aware of the
>> InterfaceStability
>> > annotation !
>> > I have updated the KIP based on your suggestions
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1265%3A+Mechanism+for+automatic+detection+of+internal+API+usage
>> > and hope that all your points have been added to it.
>> >
>> > Team - Please do let me know your thoughts.
>> > Thanks,
>> > Ashwin
>> >
>> > On Tue, Jan 6, 2026 at 6:10 AM Ismael Juma <me...@ismaeljuma.com> wrote:
>> >
>> > > Hi Ashwin,
>> > >
>> > > Improving how we specify public API and providing a mechanism for
>> > > automatic detection of internal api usage would be great. That said,
>> it's
>> > > incorrect that there is no mechanism for that today. We should update
>> the
>> > > KIP to specify the current situation correctly and the problems with
>> it.
>> > >
>> > > A summary of items to consider for the KIP:
>> > > 1. As a user, the way to know the public API is to consult the
>> project's
>> > > published javadoc: this is a "concrete, centralized definition of what
>> > > constitutes a public API".
>> > > 2. As a Kafka developer, you specify it via a block like this:
>> > > https://github.com/apache/kafka/blob/trunk/build.gradle#L2026
>> > > 3. There is no tooling that automatically detects if a public method
>> > > exposes an internal class (there was a recent KIP that was introduced
>> > > because we accidentally exposed an internal class)
>> > > 4. There is no simple mechanism for users of Kafka to configure their
>> > > build system so that usage of Kafka internal apis are flagged.
>> > > 5. There are already annotations to specify whether a class or method
>> is
>> > > stable, evolving or unstable:
>> > >
>> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/annotation/InterfaceStability.java
>> > >
>> > > Thanks,
>> > > Ismael
>> > >
>> > > On Sun, Jan 4, 2026 at 8:21 PM Ashwin via dev <de...@kafka.apache.org>
>> > > wrote:
>> > >
>> > >> Hello folks,
>> > >>
>> > >> Reviving this old thread
>> > >> https://lists.apache.org/thread/ly5ddkobr1wc07gvhwc1p0jg94qg8cxc to
>> > >> discuss
>> > >> KIP-1265.
>> > >>
>> > >> Apache Kafka lacks a concrete, centralized definition of what
>> constitutes
>> > >> a
>> > >> public API. The most relevant information currently available is
>> found
>> > >> here:
>> > >>
>> > >> Kafka Improvement
>> > >> Proposals#Whatisconsidereda%22majorchange%22thatneedsaKIP
>> > >> <
>> > >>
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaKIP
>> > >> >
>> > >>
>> > >> Without formal definition or guardrails, there is a risk that
>> builders may
>> > >> inadvertently import internal classes leading to possible build
>> breakages
>> > >> when they compile against a newer Kafka version
>> > >>
>> > >>
>> > >> Please let me know your thoughts for
>> > >>
>> > >>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1265%3A+Public+API+needs+to+be+explicitly+declared
>> > >>
>> > >> Cheers,
>> > >> Ashwin
>> > >>
>> > >
>> >
>>
>
Re: [DISCUSS] KIP-1265: Public API needs to be explicitly declared
Posted by Ashwin via dev <de...@kafka.apache.org>.
Thanks for providing feedback Stan ! Sorry I couldn't reply earlier .
I too was having second thoughts about Plugin development support which I
proposed earlier . I now feel that we should publish Gradle and Maven
plugins from Apache Kafka. Developers can then use these plugins in their
build config.
I have published the proposal at
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=406618601#KIP1265:MechanismforautomaticdetectionofinternalAPIusage-GuardrailsforPlugindevelopers
.
Please have a look and let me know what you think.
Cheers,
Ashwin
On Wed, Jan 14, 2026 at 1:53 AM Stanislav Kozlovski <
stanislavkozlovski@apache.org> wrote:
> Thanks for working on this! It's much needed, I remember many times where
> I'd catch similar misses in PR reviews in the last minute.
>
> As for helping Plugin devs, how do we precisely plan to "provide a sample
> configuration" and "publish a sample APILyzer configuration". Do we expect
> people to know where to find these things from the docs and copy them to
> their own repos?
>
> Can you think of any ways to warn Kafka users, i.e people who import
> `org.apache.kafka.Something`, of the fact they're depending on an unstable
> API too?
>
> Best,
> Stan
>
> On 2026/01/08 07:21:23 Ashwin via dev wrote:
> > Thanks for your feedback Ismael ! I was not aware of the
> InterfaceStability
> > annotation !
> > I have updated the KIP based on your suggestions
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1265%3A+Mechanism+for+automatic+detection+of+internal+API+usage
> > and hope that all your points have been added to it.
> >
> > Team - Please do let me know your thoughts.
> > Thanks,
> > Ashwin
> >
> > On Tue, Jan 6, 2026 at 6:10 AM Ismael Juma <me...@ismaeljuma.com> wrote:
> >
> > > Hi Ashwin,
> > >
> > > Improving how we specify public API and providing a mechanism for
> > > automatic detection of internal api usage would be great. That said,
> it's
> > > incorrect that there is no mechanism for that today. We should update
> the
> > > KIP to specify the current situation correctly and the problems with
> it.
> > >
> > > A summary of items to consider for the KIP:
> > > 1. As a user, the way to know the public API is to consult the
> project's
> > > published javadoc: this is a "concrete, centralized definition of what
> > > constitutes a public API".
> > > 2. As a Kafka developer, you specify it via a block like this:
> > > https://github.com/apache/kafka/blob/trunk/build.gradle#L2026
> > > 3. There is no tooling that automatically detects if a public method
> > > exposes an internal class (there was a recent KIP that was introduced
> > > because we accidentally exposed an internal class)
> > > 4. There is no simple mechanism for users of Kafka to configure their
> > > build system so that usage of Kafka internal apis are flagged.
> > > 5. There are already annotations to specify whether a class or method
> is
> > > stable, evolving or unstable:
> > >
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/annotation/InterfaceStability.java
> > >
> > > Thanks,
> > > Ismael
> > >
> > > On Sun, Jan 4, 2026 at 8:21 PM Ashwin via dev <de...@kafka.apache.org>
> > > wrote:
> > >
> > >> Hello folks,
> > >>
> > >> Reviving this old thread
> > >> https://lists.apache.org/thread/ly5ddkobr1wc07gvhwc1p0jg94qg8cxc to
> > >> discuss
> > >> KIP-1265.
> > >>
> > >> Apache Kafka lacks a concrete, centralized definition of what
> constitutes
> > >> a
> > >> public API. The most relevant information currently available is found
> > >> here:
> > >>
> > >> Kafka Improvement
> > >> Proposals#Whatisconsidereda%22majorchange%22thatneedsaKIP
> > >> <
> > >>
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaKIP
> > >> >
> > >>
> > >> Without formal definition or guardrails, there is a risk that
> builders may
> > >> inadvertently import internal classes leading to possible build
> breakages
> > >> when they compile against a newer Kafka version
> > >>
> > >>
> > >> Please let me know your thoughts for
> > >>
> > >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1265%3A+Public+API+needs+to+be+explicitly+declared
> > >>
> > >> Cheers,
> > >> Ashwin
> > >>
> > >
> >
>
Re: [DISCUSS] KIP-1265: Public API needs to be explicitly declared
Posted by Stanislav Kozlovski <st...@apache.org>.
Thanks for working on this! It's much needed, I remember many times where I'd catch similar misses in PR reviews in the last minute.
As for helping Plugin devs, how do we precisely plan to "provide a sample configuration" and "publish a sample APILyzer configuration". Do we expect people to know where to find these things from the docs and copy them to their own repos?
Can you think of any ways to warn Kafka users, i.e people who import `org.apache.kafka.Something`, of the fact they're depending on an unstable API too?
Best,
Stan
On 2026/01/08 07:21:23 Ashwin via dev wrote:
> Thanks for your feedback Ismael ! I was not aware of the InterfaceStability
> annotation !
> I have updated the KIP based on your suggestions
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1265%3A+Mechanism+for+automatic+detection+of+internal+API+usage
> and hope that all your points have been added to it.
>
> Team - Please do let me know your thoughts.
> Thanks,
> Ashwin
>
> On Tue, Jan 6, 2026 at 6:10 AM Ismael Juma <me...@ismaeljuma.com> wrote:
>
> > Hi Ashwin,
> >
> > Improving how we specify public API and providing a mechanism for
> > automatic detection of internal api usage would be great. That said, it's
> > incorrect that there is no mechanism for that today. We should update the
> > KIP to specify the current situation correctly and the problems with it.
> >
> > A summary of items to consider for the KIP:
> > 1. As a user, the way to know the public API is to consult the project's
> > published javadoc: this is a "concrete, centralized definition of what
> > constitutes a public API".
> > 2. As a Kafka developer, you specify it via a block like this:
> > https://github.com/apache/kafka/blob/trunk/build.gradle#L2026
> > 3. There is no tooling that automatically detects if a public method
> > exposes an internal class (there was a recent KIP that was introduced
> > because we accidentally exposed an internal class)
> > 4. There is no simple mechanism for users of Kafka to configure their
> > build system so that usage of Kafka internal apis are flagged.
> > 5. There are already annotations to specify whether a class or method is
> > stable, evolving or unstable:
> > https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/annotation/InterfaceStability.java
> >
> > Thanks,
> > Ismael
> >
> > On Sun, Jan 4, 2026 at 8:21 PM Ashwin via dev <de...@kafka.apache.org>
> > wrote:
> >
> >> Hello folks,
> >>
> >> Reviving this old thread
> >> https://lists.apache.org/thread/ly5ddkobr1wc07gvhwc1p0jg94qg8cxc to
> >> discuss
> >> KIP-1265.
> >>
> >> Apache Kafka lacks a concrete, centralized definition of what constitutes
> >> a
> >> public API. The most relevant information currently available is found
> >> here:
> >>
> >> Kafka Improvement
> >> Proposals#Whatisconsidereda%22majorchange%22thatneedsaKIP
> >> <
> >> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaKIP
> >> >
> >>
> >> Without formal definition or guardrails, there is a risk that builders may
> >> inadvertently import internal classes leading to possible build breakages
> >> when they compile against a newer Kafka version
> >>
> >>
> >> Please let me know your thoughts for
> >>
> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1265%3A+Public+API+needs+to+be+explicitly+declared
> >>
> >> Cheers,
> >> Ashwin
> >>
> >
>
Re: [DISCUSS] KIP-1265: Public API needs to be explicitly declared
Posted by Ashwin via dev <de...@kafka.apache.org>.
Thanks for your feedback Ismael ! I was not aware of the InterfaceStability
annotation !
I have updated the KIP based on your suggestions
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1265%3A+Mechanism+for+automatic+detection+of+internal+API+usage
and hope that all your points have been added to it.
Team - Please do let me know your thoughts.
Thanks,
Ashwin
On Tue, Jan 6, 2026 at 6:10 AM Ismael Juma <me...@ismaeljuma.com> wrote:
> Hi Ashwin,
>
> Improving how we specify public API and providing a mechanism for
> automatic detection of internal api usage would be great. That said, it's
> incorrect that there is no mechanism for that today. We should update the
> KIP to specify the current situation correctly and the problems with it.
>
> A summary of items to consider for the KIP:
> 1. As a user, the way to know the public API is to consult the project's
> published javadoc: this is a "concrete, centralized definition of what
> constitutes a public API".
> 2. As a Kafka developer, you specify it via a block like this:
> https://github.com/apache/kafka/blob/trunk/build.gradle#L2026
> 3. There is no tooling that automatically detects if a public method
> exposes an internal class (there was a recent KIP that was introduced
> because we accidentally exposed an internal class)
> 4. There is no simple mechanism for users of Kafka to configure their
> build system so that usage of Kafka internal apis are flagged.
> 5. There are already annotations to specify whether a class or method is
> stable, evolving or unstable:
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/annotation/InterfaceStability.java
>
> Thanks,
> Ismael
>
> On Sun, Jan 4, 2026 at 8:21 PM Ashwin via dev <de...@kafka.apache.org>
> wrote:
>
>> Hello folks,
>>
>> Reviving this old thread
>> https://lists.apache.org/thread/ly5ddkobr1wc07gvhwc1p0jg94qg8cxc to
>> discuss
>> KIP-1265.
>>
>> Apache Kafka lacks a concrete, centralized definition of what constitutes
>> a
>> public API. The most relevant information currently available is found
>> here:
>>
>> Kafka Improvement
>> Proposals#Whatisconsidereda%22majorchange%22thatneedsaKIP
>> <
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaKIP
>> >
>>
>> Without formal definition or guardrails, there is a risk that builders may
>> inadvertently import internal classes leading to possible build breakages
>> when they compile against a newer Kafka version
>>
>>
>> Please let me know your thoughts for
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1265%3A+Public+API+needs+to+be+explicitly+declared
>>
>> Cheers,
>> Ashwin
>>
>
Re: [DISCUSS] KIP-1265: Public API needs to be explicitly declared
Posted by Ismael Juma <me...@ismaeljuma.com>.
Hi Ashwin,
Improving how we specify public API and providing a mechanism for automatic
detection of internal api usage would be great. That said, it's incorrect
that there is no mechanism for that today. We should update the KIP to
specify the current situation correctly and the problems with it.
A summary of items to consider for the KIP:
1. As a user, the way to know the public API is to consult the project's
published javadoc: this is a "concrete, centralized definition of what
constitutes a public API".
2. As a Kafka developer, you specify it via a block like this:
https://github.com/apache/kafka/blob/trunk/build.gradle#L2026
3. There is no tooling that automatically detects if a public method
exposes an internal class (there was a recent KIP that was introduced
because we accidentally exposed an internal class)
4. There is no simple mechanism for users of Kafka to configure their build
system so that usage of Kafka internal apis are flagged.
5. There are already annotations to specify whether a class or method is
stable, evolving or unstable:
https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/annotation/InterfaceStability.java
Thanks,
Ismael
On Sun, Jan 4, 2026 at 8:21 PM Ashwin via dev <de...@kafka.apache.org> wrote:
> Hello folks,
>
> Reviving this old thread
> https://lists.apache.org/thread/ly5ddkobr1wc07gvhwc1p0jg94qg8cxc to
> discuss
> KIP-1265.
>
> Apache Kafka lacks a concrete, centralized definition of what constitutes a
> public API. The most relevant information currently available is found
> here:
>
> Kafka Improvement Proposals#Whatisconsidereda%22majorchange%22thatneedsaKIP
> <
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaKIP
> >
>
> Without formal definition or guardrails, there is a risk that builders may
> inadvertently import internal classes leading to possible build breakages
> when they compile against a newer Kafka version
>
>
> Please let me know your thoughts for
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1265%3A+Public+API+needs+to+be+explicitly+declared
>
> Cheers,
> Ashwin
>
Re: [DISCUSS] KIP-1265: Public API needs to be explicitly declared
Posted by Chia-Ping Tsai <ch...@gmail.com>.
hi Ashwin
Thanks for your reply. In order to avoid duplicating the discussion, I have
responded in the PR discussion.
Best,
Chia-Ping
Re: [DISCUSS] KIP-1265: Public API needs to be explicitly declared
Posted by Ashwin via dev <de...@kafka.apache.org>.
Hello @Chia-Ping Tsai <ch...@gmail.com> !
`protected` methods were not published as part of docsJar. But a plugin
developer may refer to its signature as part of an overridden method.
I agree with the approach you have taken in
https://urldefense.com/v3/__https://github.com/apache/kafka/pull/21337*discussion_r3574009798__;Iw!!Ayb5sqE7!uAZbLK-06CCOQ5QeCxqo-Z9DIffXzi46m-whE73S5uGoxizobJT0vxXNqsm3VftloRnanaMOmGQNUun9tHhF$
.
Should we also update the docsJar task to include protected methods as well
? This should make things more clearer.
Cheers,
Ashwin
Re: [DISCUSS] KIP-1265: Public API needs to be explicitly declared
Posted by Chia-Ping Tsai <ch...@apache.org>.
hi all
I noticed an issue regarding the "protected" modifier with this new annotation. https://github.com/apache/kafka/pull/21337#discussion_r3574009798
TL;DR: The "protected" modifier seems to be treated as part of the "public" scope for now, so we can't use it to hide [internal details] from users and we need to maintain backward compatibility?
There is another PR (https://github.com/apache/kafka/pull/20598) using the same approach (like CloseOptions), so I'd like to check the best practices for @InterfaceAudience.Public before merging it.
Best,
Chia-Ping
On 2026/01/05 04:21:10 Ashwin via dev wrote:
> Hello folks,
>
> Reviving this old thread
> https://lists.apache.org/thread/ly5ddkobr1wc07gvhwc1p0jg94qg8cxc to discuss
> KIP-1265.
>
> Apache Kafka lacks a concrete, centralized definition of what constitutes a
> public API. The most relevant information currently available is found here:
>
> Kafka Improvement Proposals#Whatisconsidereda%22majorchange%22thatneedsaKIP
> <https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaKIP>
>
> Without formal definition or guardrails, there is a risk that builders may
> inadvertently import internal classes leading to possible build breakages
> when they compile against a newer Kafka version
>
>
> Please let me know your thoughts for
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1265%3A+Public+API+needs+to+be+explicitly+declared
>
> Cheers,
> Ashwin
>