You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Ewen Cheslack-Postava <ew...@confluent.io> on 2018/08/08 01:38:26 UTC
Re: [DISCUSS] add connect-related packages to "What is considered a
"major change" that needs a KIP?"
First, I agree, updating that list would be a good idea. It's likely it
will always be a little divergent from any new additions -- the last update
was probably when the KIP page was originally created, before either
Connect or Streams existed.
However, note that we also document the exact set of public APIs, or at
least try to, in the user-facing javadocs
http://kafka.apache.org/20/javadoc/index.html?overview-summary.html If its
not in there, it likely is not public API.
In the case of Connect, the only public Java APIs are in their own module,
connect-api. Everything in the other connect modules are considered
internal implementation details.
-Ewen
On Fri, Jul 27, 2018 at 7:15 PM Chia-Ping Tsai <ch...@apache.org> wrote:
> hi Kafka
>
> There is a section[1] listing the packages which have public interfaces.
> However, it doesn't include any connect-related packages. It would be
> better to have a list used to highlight the public interfaces for
> connector. Otherwise, connector user may misuse the internal interface in
> their code base. For example, TopicAdmin[2] is a nice wrap of
> AdminClient[3] so I have applied TopicAdmin in production code. However,
> I'm not sure whether TopicAdmin is a public interface. It implies that I
> should remove it from code base since the SC of TopicAdmin may be broken in
> minor release.
>
> Not familiar with the rule of updating "Kafka Improvement Proposals" so I
> start this thread. All suggestions are welcome!
>
> [1]
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaKIP
> ?
>
> [2]
> https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/util/TopicAdmin.java
>
> [3]
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/admin/AdminClient.java
>
> --
> Chia-Ping
>
Re: [DISCUSS] add connect-related packages to "What is considered a
"major change" that needs a KIP?"
Posted by Ron Dagostino <rn...@gmail.com>.
Oops, ignore that messy shell command -- it misses the streams packages.
If anyone knows how to use gradle to get the actual list of packages I
would be interested in knowing the command.
Ron
On Wed, Aug 8, 2018 at 9:59 AM Ron Dagostino <rn...@gmail.com> wrote:
> Yes, I have recently been wondering what exactly the set of public-facing
> APIs really is. If it is fact defined by the Javadoc then the KIP info
> page should state this. In case anyone finds it helpful, see below for one
> (admittedly imperfect) way to see the list of packages. There is probably
> a much better way using gradle, but that way escapes me at the moment.
>
> Ron
>
> $ *grep -A 20 '[ \t]*javadoc {' build.gradle | grep '[ \t]*include[
> \t]*"\*\*.*/\*"' | sed 's/^.*"\*\*\///' | sed 's/\/\*"//' | sed 's/\//\./g'
> | sort*
> org.apache.kafka.clients.admin
> org.apache.kafka.clients.consumer
> org.apache.kafka.clients.producer
> org.apache.kafka.common
> org.apache.kafka.common.acl
> org.apache.kafka.common.annotation
> org.apache.kafka.common.config
> org.apache.kafka.common.config.provider
> org.apache.kafka.common.errors
> org.apache.kafka.common.header
> org.apache.kafka.common.resource
> org.apache.kafka.common.security.auth
> org.apache.kafka.common.security.oauthbearer
> org.apache.kafka.common.security.plain
> org.apache.kafka.common.security.scram
> org.apache.kafka.common.security.token.delegation
> org.apache.kafka.common.serialization
> org.apache.kafka.server.policy
> org.apache.kafka.server.quota
>
> On Wed, Aug 8, 2018 at 6:18 AM Chia-Ping Tsai <ch...@apache.org> wrote:
>
>> hi Ewen
>>
>> > However, note that we also document the exact set of public APIs, or at
>> > least try to, in the user-facing javadocs
>> > http://kafka.apache.org/20/javadoc/index.html?overview-summary.html If
>> its
>> > not in there, it likely is not public API.
>>
>> Thanks for the information! The user-facing javadocs[1] is clear enough
>> to me.
>>
>> --
>> chia-ping
>>
>> On 2018/08/08 01:38:26, Ewen Cheslack-Postava <ew...@confluent.io> wrote:
>> > First, I agree, updating that list would be a good idea. It's likely it
>> > will always be a little divergent from any new additions -- the last
>> update
>> > was probably when the KIP page was originally created, before either
>> > Connect or Streams existed.
>> >
>> > However, note that we also document the exact set of public APIs, or at
>> > least try to, in the user-facing javadocs
>> > http://kafka.apache.org/20/javadoc/index.html?overview-summary.html If
>> its
>> > not in there, it likely is not public API.
>> >
>> > In the case of Connect, the only public Java APIs are in their own
>> module,
>> > connect-api. Everything in the other connect modules are considered
>> > internal implementation details.
>> >
>> > -Ewen
>> >
>> >
>> > On Fri, Jul 27, 2018 at 7:15 PM Chia-Ping Tsai <ch...@apache.org>
>> wrote:
>> >
>> > > hi Kafka
>> > >
>> > > There is a section[1] listing the packages which have public
>> interfaces.
>> > > However, it doesn't include any connect-related packages. It would be
>> > > better to have a list used to highlight the public interfaces for
>> > > connector. Otherwise, connector user may misuse the internal
>> interface in
>> > > their code base. For example, TopicAdmin[2] is a nice wrap of
>> > > AdminClient[3] so I have applied TopicAdmin in production code.
>> However,
>> > > I'm not sure whether TopicAdmin is a public interface. It implies
>> that I
>> > > should remove it from code base since the SC of TopicAdmin may be
>> broken in
>> > > minor release.
>> > >
>> > > Not familiar with the rule of updating "Kafka Improvement Proposals"
>> so I
>> > > start this thread. All suggestions are welcome!
>> > >
>> > > [1]
>> > >
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaKIP
>> > > ?
>> > >
>> > > [2]
>> > >
>> https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/util/TopicAdmin.java
>> > >
>> > > [3]
>> > >
>> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/admin/AdminClient.java
>> > >
>> > > --
>> > > Chia-Ping
>> > >
>> >
>>
>
Re: [DISCUSS] add connect-related packages to "What is considered a
"major change" that needs a KIP?"
Posted by Ron Dagostino <rn...@gmail.com>.
Yes, I have recently been wondering what exactly the set of public-facing
APIs really is. If it is fact defined by the Javadoc then the KIP info
page should state this. In case anyone finds it helpful, see below for one
(admittedly imperfect) way to see the list of packages. There is probably
a much better way using gradle, but that way escapes me at the moment.
Ron
$ *grep -A 20 '[ \t]*javadoc {' build.gradle | grep '[ \t]*include[
\t]*"\*\*.*/\*"' | sed 's/^.*"\*\*\///' | sed 's/\/\*"//' | sed 's/\//\./g'
| sort*
org.apache.kafka.clients.admin
org.apache.kafka.clients.consumer
org.apache.kafka.clients.producer
org.apache.kafka.common
org.apache.kafka.common.acl
org.apache.kafka.common.annotation
org.apache.kafka.common.config
org.apache.kafka.common.config.provider
org.apache.kafka.common.errors
org.apache.kafka.common.header
org.apache.kafka.common.resource
org.apache.kafka.common.security.auth
org.apache.kafka.common.security.oauthbearer
org.apache.kafka.common.security.plain
org.apache.kafka.common.security.scram
org.apache.kafka.common.security.token.delegation
org.apache.kafka.common.serialization
org.apache.kafka.server.policy
org.apache.kafka.server.quota
On Wed, Aug 8, 2018 at 6:18 AM Chia-Ping Tsai <ch...@apache.org> wrote:
> hi Ewen
>
> > However, note that we also document the exact set of public APIs, or at
> > least try to, in the user-facing javadocs
> > http://kafka.apache.org/20/javadoc/index.html?overview-summary.html If
> its
> > not in there, it likely is not public API.
>
> Thanks for the information! The user-facing javadocs[1] is clear enough to
> me.
>
> --
> chia-ping
>
> On 2018/08/08 01:38:26, Ewen Cheslack-Postava <ew...@confluent.io> wrote:
> > First, I agree, updating that list would be a good idea. It's likely it
> > will always be a little divergent from any new additions -- the last
> update
> > was probably when the KIP page was originally created, before either
> > Connect or Streams existed.
> >
> > However, note that we also document the exact set of public APIs, or at
> > least try to, in the user-facing javadocs
> > http://kafka.apache.org/20/javadoc/index.html?overview-summary.html If
> its
> > not in there, it likely is not public API.
> >
> > In the case of Connect, the only public Java APIs are in their own
> module,
> > connect-api. Everything in the other connect modules are considered
> > internal implementation details.
> >
> > -Ewen
> >
> >
> > On Fri, Jul 27, 2018 at 7:15 PM Chia-Ping Tsai <ch...@apache.org>
> wrote:
> >
> > > hi Kafka
> > >
> > > There is a section[1] listing the packages which have public
> interfaces.
> > > However, it doesn't include any connect-related packages. It would be
> > > better to have a list used to highlight the public interfaces for
> > > connector. Otherwise, connector user may misuse the internal interface
> in
> > > their code base. For example, TopicAdmin[2] is a nice wrap of
> > > AdminClient[3] so I have applied TopicAdmin in production code.
> However,
> > > I'm not sure whether TopicAdmin is a public interface. It implies that
> I
> > > should remove it from code base since the SC of TopicAdmin may be
> broken in
> > > minor release.
> > >
> > > Not familiar with the rule of updating "Kafka Improvement Proposals"
> so I
> > > start this thread. All suggestions are welcome!
> > >
> > > [1]
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaKIP
> > > ?
> > >
> > > [2]
> > >
> https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/util/TopicAdmin.java
> > >
> > > [3]
> > >
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/admin/AdminClient.java
> > >
> > > --
> > > Chia-Ping
> > >
> >
>
Re: [DISCUSS] add connect-related packages to "What is considered a "major change" that needs a KIP?"
Posted by Chia-Ping Tsai <ch...@apache.org>.
hi Ewen
> However, note that we also document the exact set of public APIs, or at
> least try to, in the user-facing javadocs
> http://kafka.apache.org/20/javadoc/index.html?overview-summary.html If its
> not in there, it likely is not public API.
Thanks for the information! The user-facing javadocs[1] is clear enough to me.
--
chia-ping
On 2018/08/08 01:38:26, Ewen Cheslack-Postava <ew...@confluent.io> wrote:
> First, I agree, updating that list would be a good idea. It's likely it
> will always be a little divergent from any new additions -- the last update
> was probably when the KIP page was originally created, before either
> Connect or Streams existed.
>
> However, note that we also document the exact set of public APIs, or at
> least try to, in the user-facing javadocs
> http://kafka.apache.org/20/javadoc/index.html?overview-summary.html If its
> not in there, it likely is not public API.
>
> In the case of Connect, the only public Java APIs are in their own module,
> connect-api. Everything in the other connect modules are considered
> internal implementation details.
>
> -Ewen
>
>
> On Fri, Jul 27, 2018 at 7:15 PM Chia-Ping Tsai <ch...@apache.org> wrote:
>
> > hi Kafka
> >
> > There is a section[1] listing the packages which have public interfaces.
> > However, it doesn't include any connect-related packages. It would be
> > better to have a list used to highlight the public interfaces for
> > connector. Otherwise, connector user may misuse the internal interface in
> > their code base. For example, TopicAdmin[2] is a nice wrap of
> > AdminClient[3] so I have applied TopicAdmin in production code. However,
> > I'm not sure whether TopicAdmin is a public interface. It implies that I
> > should remove it from code base since the SC of TopicAdmin may be broken in
> > minor release.
> >
> > Not familiar with the rule of updating "Kafka Improvement Proposals" so I
> > start this thread. All suggestions are welcome!
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaKIP
> > ?
> >
> > [2]
> > https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/util/TopicAdmin.java
> >
> > [3]
> > https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/admin/AdminClient.java
> >
> > --
> > Chia-Ping
> >
>