You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Grant Henke <gh...@cloudera.com> on 2016/04/08 23:41:01 UTC
[VOTE] KIP-4 Metadata Schema (Round 2)
I would like to re-initiate the voting process for the "KIP-4 Metadata
Schema changes". This is not a vote for all of KIP-4, but specifically for
the metadata changes. I have included the exact changes below for clarity:
>
> Metadata Request (version 1)
>
>
>
> MetadataRequest => [topics]
>
> Stays the same as version 0 however behavior changes.
> In version 0 there was no way to request no topics, and and empty list
> signified all topics.
> In version 1 a null topics list (size -1 on the wire) will indicate that a
> user wants *ALL* topic metadata. Compared to an empty list (size 0) which
> indicates metadata for *NO* topics should be returned.
> Metadata Response (version 1)
>
>
>
> MetadataResponse => [brokers] controllerId [topic_metadata]
> brokers => node_id host port rack
> node_id => INT32
> host => STRING
> port => INT32
> rack => NULLABLE_STRING
> controllerId => INT32
> topic_metadata => topic_error_code topic is_internal [partition_metadata]
> topic_error_code => INT16
> topic => STRING
> is_internal => BOOLEAN
> partition_metadata => partition_error_code partition_id leader [replicas] [isr]
> partition_error_code => INT16
> partition_id => INT32
> leader => INT32
> replicas => INT32
> isr => INT32
>
> Adds rack, controller_id, and is_internal to the version 0 response.
>
The KIP is available here for reference (linked to the Metadata schema
section):
*https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
<https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema>*
A pull request is available implementing the proposed changes here:
https://github.com/apache/kafka/pull/1095
Here are some links to past discussions on the mailing list:
http://search-hadoop.com/m/uyzND1pd4T52H1m0u1&subj=Re+KIP+4+Wiki+Update
http://search-hadoop.com/m/uyzND1J2IXeSNXAT&subj=Metadata+and+ACLs+wire+protocol+review+KIP+4+
Here is the previous vote discussion (please take a look and discuss there):
http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
Thank you,
Grant
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [VOTE] KIP-4 Metadata Schema (Round 2)
Posted by Ismael Juma <is...@juma.me.uk>.
+1 (non-binding)
On Fri, Apr 8, 2016 at 10:41 PM, Grant Henke <gh...@cloudera.com> wrote:
> I would like to re-initiate the voting process for the "KIP-4 Metadata
> Schema changes". This is not a vote for all of KIP-4, but specifically for
> the metadata changes. I have included the exact changes below for clarity:
> >
> > Metadata Request (version 1)
> >
> >
> >
> > MetadataRequest => [topics]
> >
> > Stays the same as version 0 however behavior changes.
> > In version 0 there was no way to request no topics, and and empty list
> > signified all topics.
> > In version 1 a null topics list (size -1 on the wire) will indicate that
> a
> > user wants *ALL* topic metadata. Compared to an empty list (size 0) which
> > indicates metadata for *NO* topics should be returned.
> > Metadata Response (version 1)
> >
> >
> >
> > MetadataResponse => [brokers] controllerId [topic_metadata]
> > brokers => node_id host port rack
> > node_id => INT32
> > host => STRING
> > port => INT32
> > rack => NULLABLE_STRING
> > controllerId => INT32
> > topic_metadata => topic_error_code topic is_internal
> [partition_metadata]
> > topic_error_code => INT16
> > topic => STRING
> > is_internal => BOOLEAN
> > partition_metadata => partition_error_code partition_id leader
> [replicas] [isr]
> > partition_error_code => INT16
> > partition_id => INT32
> > leader => INT32
> > replicas => INT32
> > isr => INT32
> >
> > Adds rack, controller_id, and is_internal to the version 0 response.
> >
>
> The KIP is available here for reference (linked to the Metadata schema
> section):
> *
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> >*
>
> A pull request is available implementing the proposed changes here:
> https://github.com/apache/kafka/pull/1095
>
> Here are some links to past discussions on the mailing list:
> http://search-hadoop.com/m/uyzND1pd4T52H1m0u1&subj=Re+KIP+4+Wiki+Update
>
> http://search-hadoop.com/m/uyzND1J2IXeSNXAT&subj=Metadata+and+ACLs+wire+protocol+review+KIP+4+
>
> Here is the previous vote discussion (please take a look and discuss
> there):
>
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
>
> Thank you,
> Grant
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [VOTE] KIP-4 Metadata Schema (Round 2)
Posted by Ismael Juma <is...@juma.me.uk>.
Thanks!
Ismael
On Fri, Apr 15, 2016 at 3:36 PM, Grant Henke <gh...@cloudera.com> wrote:
> I added a row on the main KIP wiki page to indicate the metadata changes
> will be in 0.10 and I also update the KIP-4 wiki too.
>
> Thanks,
> Grant
>
> On Fri, Apr 15, 2016 at 9:27 AM, Ismael Juma <is...@juma.me.uk> wrote:
>
> > Third time is a charm. :) Can you please update the wiki page below to
> > include this KIP in the adopted table and 0.10.0.0 as the release?
> >
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> >
> > I know we already have KIP-4 in that table, but I think it's useful for
> > people to know that KIP-4 Metadata is being included in 0.10.0.0 (whereas
> > the rest is not).
> >
> > Ismael
> >
> > On Fri, Apr 15, 2016 at 3:19 PM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> > > Thanks to all who voted. The KIP-4 Metadata changes passed with +3
> > > (binding), and +4 (non-binding).
> > >
> > > There is a patch available for review here:
> > > https://github.com/apache/kafka/pull/1095
> > >
> > > On Tue, Apr 12, 2016 at 11:23 AM, Jason Gustafson <ja...@confluent.io>
> > > wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > On Mon, Apr 11, 2016 at 6:21 PM, Ismael Juma <is...@juma.me.uk>
> > wrote:
> > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > Ismael
> > > > >
> > > > > On Tue, Apr 12, 2016 at 2:19 AM, Jun Rao <ju...@confluent.io> wrote:
> > > > >
> > > > > > Grant,
> > > > > >
> > > > > > Thanks for the updated version. +1 from me.
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > > On Mon, Apr 11, 2016 at 10:42 AM, Grant Henke <
> ghenke@cloudera.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Based on the discussion in the previous vote thread
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> > > > > > > >
> > > > > > > I also would like to include a behavior change to the
> > > > > MetadataResponse. I
> > > > > > > have update the wiki
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > > > > >
> > > > > > > and pull request <https://github.com/apache/kafka/pull/1095>
> to
> > > > > include
> > > > > > > this change.
> > > > > > >
> > > > > > > The change as described on the wiki is:
> > > > > > >
> > > > > > > > The behavior of the replicas and isr arrays will be changed
> in
> > > > order
> > > > > to
> > > > > > > > support the admin tools, and better represent the state of
> the
> > > > > cluster:
> > > > > > > >
> > > > > > > > - In version 0, if a broker is down the replicas and isr
> > array
> > > > > will
> > > > > > > > omit the brokers entry and add a REPLICA_NOT_AVAILABLE
> error
> > > > code.
> > > > > > > > - In version 1, no error code will be set and a the broker
> > id
> > > > will
> > > > > > be
> > > > > > > > included in the replicas and isr array.
> > > > > > > > - Note: A user can still detect if the replica is not
> > > > > available,
> > > > > > by
> > > > > > > > checking if the broker is in the returned broker list.
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > Being optimistic that this doesn't require to much discussion,
> I
> > > > would
> > > > > > like
> > > > > > > to re-start the voting process on this thread. If more
> discussion
> > > is
> > > > > > > needed, please don't hesitate to bring it up here.
> > > > > > >
> > > > > > > Ismael, Gwen, Guozhang could you please review and revote based
> > on
> > > > the
> > > > > > > changes.
> > > > > > >
> > > > > > > Thank you,
> > > > > > > Grant
> > > > > > >
> > > > > > > On Sat, Apr 9, 2016 at 1:03 PM, Guozhang Wang <
> > wangguoz@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > > +1
> > > > > > > >
> > > > > > > > On Fri, Apr 8, 2016 at 4:36 PM, Gwen Shapira <
> > gwen@confluent.io>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > +1
> > > > > > > > >
> > > > > > > > > On Fri, Apr 8, 2016 at 2:41 PM, Grant Henke <
> > > ghenke@cloudera.com
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > I would like to re-initiate the voting process for the
> > "KIP-4
> > > > > > > Metadata
> > > > > > > > > > Schema changes". This is not a vote for all of KIP-4, but
> > > > > > > specifically
> > > > > > > > > for
> > > > > > > > > > the metadata changes. I have included the exact changes
> > below
> > > > for
> > > > > > > > > clarity:
> > > > > > > > > > >
> > > > > > > > > > > Metadata Request (version 1)
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > MetadataRequest => [topics]
> > > > > > > > > > >
> > > > > > > > > > > Stays the same as version 0 however behavior changes.
> > > > > > > > > > > In version 0 there was no way to request no topics, and
> > and
> > > > > empty
> > > > > > > > list
> > > > > > > > > > > signified all topics.
> > > > > > > > > > > In version 1 a null topics list (size -1 on the wire)
> > will
> > > > > > indicate
> > > > > > > > > that
> > > > > > > > > > a
> > > > > > > > > > > user wants *ALL* topic metadata. Compared to an empty
> > list
> > > > > (size
> > > > > > 0)
> > > > > > > > > which
> > > > > > > > > > > indicates metadata for *NO* topics should be returned.
> > > > > > > > > > > Metadata Response (version 1)
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > MetadataResponse => [brokers] controllerId
> > [topic_metadata]
> > > > > > > > > > > brokers => node_id host port rack
> > > > > > > > > > > node_id => INT32
> > > > > > > > > > > host => STRING
> > > > > > > > > > > port => INT32
> > > > > > > > > > > rack => NULLABLE_STRING
> > > > > > > > > > > controllerId => INT32
> > > > > > > > > > > topic_metadata => topic_error_code topic is_internal
> > > > > > > > > > [partition_metadata]
> > > > > > > > > > > topic_error_code => INT16
> > > > > > > > > > > topic => STRING
> > > > > > > > > > > is_internal => BOOLEAN
> > > > > > > > > > > partition_metadata => partition_error_code
> > partition_id
> > > > > > leader
> > > > > > > > > > [replicas] [isr]
> > > > > > > > > > > partition_error_code => INT16
> > > > > > > > > > > partition_id => INT32
> > > > > > > > > > > leader => INT32
> > > > > > > > > > > replicas => INT32
> > > > > > > > > > > isr => INT32
> > > > > > > > > > >
> > > > > > > > > > > Adds rack, controller_id, and is_internal to the
> version
> > 0
> > > > > > > response.
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > The KIP is available here for reference (linked to the
> > > Metadata
> > > > > > > schema
> > > > > > > > > > section):
> > > > > > > > > > *
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > > > > > > > <
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > > > > > > > >*
> > > > > > > > > >
> > > > > > > > > > A pull request is available implementing the proposed
> > changes
> > > > > here:
> > > > > > > > > > https://github.com/apache/kafka/pull/1095
> > > > > > > > > >
> > > > > > > > > > Here are some links to past discussions on the mailing
> > list:
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > http://search-hadoop.com/m/uyzND1pd4T52H1m0u1&subj=Re+KIP+4+Wiki+Update
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1J2IXeSNXAT&subj=Metadata+and+ACLs+wire+protocol+review+KIP+4+
> > > > > > > > > >
> > > > > > > > > > Here is the previous vote discussion (please take a look
> > and
> > > > > > discuss
> > > > > > > > > > there):
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> > > > > > > > > >
> > > > > > > > > > Thank you,
> > > > > > > > > > Grant
> > > > > > > > > > --
> > > > > > > > > > Grant Henke
> > > > > > > > > > Software Engineer | Cloudera
> > > > > > > > > > grant@cloudera.com | twitter.com/gchenke |
> > > > > > > linkedin.com/in/granthenke
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > -- Guozhang
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Grant Henke
> > > > > > > Software Engineer | Cloudera
> > > > > > > grant@cloudera.com | twitter.com/gchenke |
> > > > linkedin.com/in/granthenke
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [VOTE] KIP-4 Metadata Schema (Round 2)
Posted by Grant Henke <gh...@cloudera.com>.
I added a row on the main KIP wiki page to indicate the metadata changes
will be in 0.10 and I also update the KIP-4 wiki too.
Thanks,
Grant
On Fri, Apr 15, 2016 at 9:27 AM, Ismael Juma <is...@juma.me.uk> wrote:
> Third time is a charm. :) Can you please update the wiki page below to
> include this KIP in the adopted table and 0.10.0.0 as the release?
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>
> I know we already have KIP-4 in that table, but I think it's useful for
> people to know that KIP-4 Metadata is being included in 0.10.0.0 (whereas
> the rest is not).
>
> Ismael
>
> On Fri, Apr 15, 2016 at 3:19 PM, Grant Henke <gh...@cloudera.com> wrote:
>
> > Thanks to all who voted. The KIP-4 Metadata changes passed with +3
> > (binding), and +4 (non-binding).
> >
> > There is a patch available for review here:
> > https://github.com/apache/kafka/pull/1095
> >
> > On Tue, Apr 12, 2016 at 11:23 AM, Jason Gustafson <ja...@confluent.io>
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > On Mon, Apr 11, 2016 at 6:21 PM, Ismael Juma <is...@juma.me.uk>
> wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > Ismael
> > > >
> > > > On Tue, Apr 12, 2016 at 2:19 AM, Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > Grant,
> > > > >
> > > > > Thanks for the updated version. +1 from me.
> > > > >
> > > > > Jun
> > > > >
> > > > > On Mon, Apr 11, 2016 at 10:42 AM, Grant Henke <ghenke@cloudera.com
> >
> > > > wrote:
> > > > >
> > > > > > Based on the discussion in the previous vote thread
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> > > > > > >
> > > > > > I also would like to include a behavior change to the
> > > > MetadataResponse. I
> > > > > > have update the wiki
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > > > >
> > > > > > and pull request <https://github.com/apache/kafka/pull/1095> to
> > > > include
> > > > > > this change.
> > > > > >
> > > > > > The change as described on the wiki is:
> > > > > >
> > > > > > > The behavior of the replicas and isr arrays will be changed in
> > > order
> > > > to
> > > > > > > support the admin tools, and better represent the state of the
> > > > cluster:
> > > > > > >
> > > > > > > - In version 0, if a broker is down the replicas and isr
> array
> > > > will
> > > > > > > omit the brokers entry and add a REPLICA_NOT_AVAILABLE error
> > > code.
> > > > > > > - In version 1, no error code will be set and a the broker
> id
> > > will
> > > > > be
> > > > > > > included in the replicas and isr array.
> > > > > > > - Note: A user can still detect if the replica is not
> > > > available,
> > > > > by
> > > > > > > checking if the broker is in the returned broker list.
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > Being optimistic that this doesn't require to much discussion, I
> > > would
> > > > > like
> > > > > > to re-start the voting process on this thread. If more discussion
> > is
> > > > > > needed, please don't hesitate to bring it up here.
> > > > > >
> > > > > > Ismael, Gwen, Guozhang could you please review and revote based
> on
> > > the
> > > > > > changes.
> > > > > >
> > > > > > Thank you,
> > > > > > Grant
> > > > > >
> > > > > > On Sat, Apr 9, 2016 at 1:03 PM, Guozhang Wang <
> wangguoz@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > +1
> > > > > > >
> > > > > > > On Fri, Apr 8, 2016 at 4:36 PM, Gwen Shapira <
> gwen@confluent.io>
> > > > > wrote:
> > > > > > >
> > > > > > > > +1
> > > > > > > >
> > > > > > > > On Fri, Apr 8, 2016 at 2:41 PM, Grant Henke <
> > ghenke@cloudera.com
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > I would like to re-initiate the voting process for the
> "KIP-4
> > > > > > Metadata
> > > > > > > > > Schema changes". This is not a vote for all of KIP-4, but
> > > > > > specifically
> > > > > > > > for
> > > > > > > > > the metadata changes. I have included the exact changes
> below
> > > for
> > > > > > > > clarity:
> > > > > > > > > >
> > > > > > > > > > Metadata Request (version 1)
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > MetadataRequest => [topics]
> > > > > > > > > >
> > > > > > > > > > Stays the same as version 0 however behavior changes.
> > > > > > > > > > In version 0 there was no way to request no topics, and
> and
> > > > empty
> > > > > > > list
> > > > > > > > > > signified all topics.
> > > > > > > > > > In version 1 a null topics list (size -1 on the wire)
> will
> > > > > indicate
> > > > > > > > that
> > > > > > > > > a
> > > > > > > > > > user wants *ALL* topic metadata. Compared to an empty
> list
> > > > (size
> > > > > 0)
> > > > > > > > which
> > > > > > > > > > indicates metadata for *NO* topics should be returned.
> > > > > > > > > > Metadata Response (version 1)
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > MetadataResponse => [brokers] controllerId
> [topic_metadata]
> > > > > > > > > > brokers => node_id host port rack
> > > > > > > > > > node_id => INT32
> > > > > > > > > > host => STRING
> > > > > > > > > > port => INT32
> > > > > > > > > > rack => NULLABLE_STRING
> > > > > > > > > > controllerId => INT32
> > > > > > > > > > topic_metadata => topic_error_code topic is_internal
> > > > > > > > > [partition_metadata]
> > > > > > > > > > topic_error_code => INT16
> > > > > > > > > > topic => STRING
> > > > > > > > > > is_internal => BOOLEAN
> > > > > > > > > > partition_metadata => partition_error_code
> partition_id
> > > > > leader
> > > > > > > > > [replicas] [isr]
> > > > > > > > > > partition_error_code => INT16
> > > > > > > > > > partition_id => INT32
> > > > > > > > > > leader => INT32
> > > > > > > > > > replicas => INT32
> > > > > > > > > > isr => INT32
> > > > > > > > > >
> > > > > > > > > > Adds rack, controller_id, and is_internal to the version
> 0
> > > > > > response.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > The KIP is available here for reference (linked to the
> > Metadata
> > > > > > schema
> > > > > > > > > section):
> > > > > > > > > *
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > > > > > > <
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > > > > > > >*
> > > > > > > > >
> > > > > > > > > A pull request is available implementing the proposed
> changes
> > > > here:
> > > > > > > > > https://github.com/apache/kafka/pull/1095
> > > > > > > > >
> > > > > > > > > Here are some links to past discussions on the mailing
> list:
> > > > > > > > >
> > > > > > >
> > > > >
> > >
> http://search-hadoop.com/m/uyzND1pd4T52H1m0u1&subj=Re+KIP+4+Wiki+Update
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1J2IXeSNXAT&subj=Metadata+and+ACLs+wire+protocol+review+KIP+4+
> > > > > > > > >
> > > > > > > > > Here is the previous vote discussion (please take a look
> and
> > > > > discuss
> > > > > > > > > there):
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> > > > > > > > >
> > > > > > > > > Thank you,
> > > > > > > > > Grant
> > > > > > > > > --
> > > > > > > > > Grant Henke
> > > > > > > > > Software Engineer | Cloudera
> > > > > > > > > grant@cloudera.com | twitter.com/gchenke |
> > > > > > linkedin.com/in/granthenke
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > -- Guozhang
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Grant Henke
> > > > > > Software Engineer | Cloudera
> > > > > > grant@cloudera.com | twitter.com/gchenke |
> > > linkedin.com/in/granthenke
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [VOTE] KIP-4 Metadata Schema (Round 2)
Posted by Ismael Juma <is...@juma.me.uk>.
Third time is a charm. :) Can you please update the wiki page below to
include this KIP in the adopted table and 0.10.0.0 as the release?
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
I know we already have KIP-4 in that table, but I think it's useful for
people to know that KIP-4 Metadata is being included in 0.10.0.0 (whereas
the rest is not).
Ismael
On Fri, Apr 15, 2016 at 3:19 PM, Grant Henke <gh...@cloudera.com> wrote:
> Thanks to all who voted. The KIP-4 Metadata changes passed with +3
> (binding), and +4 (non-binding).
>
> There is a patch available for review here:
> https://github.com/apache/kafka/pull/1095
>
> On Tue, Apr 12, 2016 at 11:23 AM, Jason Gustafson <ja...@confluent.io>
> wrote:
>
> > +1 (non-binding)
> >
> > On Mon, Apr 11, 2016 at 6:21 PM, Ismael Juma <is...@juma.me.uk> wrote:
> >
> > > +1 (non-binding)
> > >
> > > Ismael
> > >
> > > On Tue, Apr 12, 2016 at 2:19 AM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Grant,
> > > >
> > > > Thanks for the updated version. +1 from me.
> > > >
> > > > Jun
> > > >
> > > > On Mon, Apr 11, 2016 at 10:42 AM, Grant Henke <gh...@cloudera.com>
> > > wrote:
> > > >
> > > > > Based on the discussion in the previous vote thread
> > > > > <
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> > > > > >
> > > > > I also would like to include a behavior change to the
> > > MetadataResponse. I
> > > > > have update the wiki
> > > > > <
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > > >
> > > > > and pull request <https://github.com/apache/kafka/pull/1095> to
> > > include
> > > > > this change.
> > > > >
> > > > > The change as described on the wiki is:
> > > > >
> > > > > > The behavior of the replicas and isr arrays will be changed in
> > order
> > > to
> > > > > > support the admin tools, and better represent the state of the
> > > cluster:
> > > > > >
> > > > > > - In version 0, if a broker is down the replicas and isr array
> > > will
> > > > > > omit the brokers entry and add a REPLICA_NOT_AVAILABLE error
> > code.
> > > > > > - In version 1, no error code will be set and a the broker id
> > will
> > > > be
> > > > > > included in the replicas and isr array.
> > > > > > - Note: A user can still detect if the replica is not
> > > available,
> > > > by
> > > > > > checking if the broker is in the returned broker list.
> > > > > >
> > > > > >
> > > > >
> > > > > Being optimistic that this doesn't require to much discussion, I
> > would
> > > > like
> > > > > to re-start the voting process on this thread. If more discussion
> is
> > > > > needed, please don't hesitate to bring it up here.
> > > > >
> > > > > Ismael, Gwen, Guozhang could you please review and revote based on
> > the
> > > > > changes.
> > > > >
> > > > > Thank you,
> > > > > Grant
> > > > >
> > > > > On Sat, Apr 9, 2016 at 1:03 PM, Guozhang Wang <wa...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > +1
> > > > > >
> > > > > > On Fri, Apr 8, 2016 at 4:36 PM, Gwen Shapira <gw...@confluent.io>
> > > > wrote:
> > > > > >
> > > > > > > +1
> > > > > > >
> > > > > > > On Fri, Apr 8, 2016 at 2:41 PM, Grant Henke <
> ghenke@cloudera.com
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > I would like to re-initiate the voting process for the "KIP-4
> > > > > Metadata
> > > > > > > > Schema changes". This is not a vote for all of KIP-4, but
> > > > > specifically
> > > > > > > for
> > > > > > > > the metadata changes. I have included the exact changes below
> > for
> > > > > > > clarity:
> > > > > > > > >
> > > > > > > > > Metadata Request (version 1)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > MetadataRequest => [topics]
> > > > > > > > >
> > > > > > > > > Stays the same as version 0 however behavior changes.
> > > > > > > > > In version 0 there was no way to request no topics, and and
> > > empty
> > > > > > list
> > > > > > > > > signified all topics.
> > > > > > > > > In version 1 a null topics list (size -1 on the wire) will
> > > > indicate
> > > > > > > that
> > > > > > > > a
> > > > > > > > > user wants *ALL* topic metadata. Compared to an empty list
> > > (size
> > > > 0)
> > > > > > > which
> > > > > > > > > indicates metadata for *NO* topics should be returned.
> > > > > > > > > Metadata Response (version 1)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > MetadataResponse => [brokers] controllerId [topic_metadata]
> > > > > > > > > brokers => node_id host port rack
> > > > > > > > > node_id => INT32
> > > > > > > > > host => STRING
> > > > > > > > > port => INT32
> > > > > > > > > rack => NULLABLE_STRING
> > > > > > > > > controllerId => INT32
> > > > > > > > > topic_metadata => topic_error_code topic is_internal
> > > > > > > > [partition_metadata]
> > > > > > > > > topic_error_code => INT16
> > > > > > > > > topic => STRING
> > > > > > > > > is_internal => BOOLEAN
> > > > > > > > > partition_metadata => partition_error_code partition_id
> > > > leader
> > > > > > > > [replicas] [isr]
> > > > > > > > > partition_error_code => INT16
> > > > > > > > > partition_id => INT32
> > > > > > > > > leader => INT32
> > > > > > > > > replicas => INT32
> > > > > > > > > isr => INT32
> > > > > > > > >
> > > > > > > > > Adds rack, controller_id, and is_internal to the version 0
> > > > > response.
> > > > > > > > >
> > > > > > > >
> > > > > > > > The KIP is available here for reference (linked to the
> Metadata
> > > > > schema
> > > > > > > > section):
> > > > > > > > *
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > > > > > <
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > > > > > >*
> > > > > > > >
> > > > > > > > A pull request is available implementing the proposed changes
> > > here:
> > > > > > > > https://github.com/apache/kafka/pull/1095
> > > > > > > >
> > > > > > > > Here are some links to past discussions on the mailing list:
> > > > > > > >
> > > > > >
> > > >
> > http://search-hadoop.com/m/uyzND1pd4T52H1m0u1&subj=Re+KIP+4+Wiki+Update
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1J2IXeSNXAT&subj=Metadata+and+ACLs+wire+protocol+review+KIP+4+
> > > > > > > >
> > > > > > > > Here is the previous vote discussion (please take a look and
> > > > discuss
> > > > > > > > there):
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> > > > > > > >
> > > > > > > > Thank you,
> > > > > > > > Grant
> > > > > > > > --
> > > > > > > > Grant Henke
> > > > > > > > Software Engineer | Cloudera
> > > > > > > > grant@cloudera.com | twitter.com/gchenke |
> > > > > linkedin.com/in/granthenke
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > -- Guozhang
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Grant Henke
> > > > > Software Engineer | Cloudera
> > > > > grant@cloudera.com | twitter.com/gchenke |
> > linkedin.com/in/granthenke
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [VOTE] KIP-4 Metadata Schema (Round 2)
Posted by Grant Henke <gh...@cloudera.com>.
Thanks to all who voted. The KIP-4 Metadata changes passed with +3
(binding), and +4 (non-binding).
There is a patch available for review here:
https://github.com/apache/kafka/pull/1095
On Tue, Apr 12, 2016 at 11:23 AM, Jason Gustafson <ja...@confluent.io>
wrote:
> +1 (non-binding)
>
> On Mon, Apr 11, 2016 at 6:21 PM, Ismael Juma <is...@juma.me.uk> wrote:
>
> > +1 (non-binding)
> >
> > Ismael
> >
> > On Tue, Apr 12, 2016 at 2:19 AM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Grant,
> > >
> > > Thanks for the updated version. +1 from me.
> > >
> > > Jun
> > >
> > > On Mon, Apr 11, 2016 at 10:42 AM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > >
> > > > Based on the discussion in the previous vote thread
> > > > <
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> > > > >
> > > > I also would like to include a behavior change to the
> > MetadataResponse. I
> > > > have update the wiki
> > > > <
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > >
> > > > and pull request <https://github.com/apache/kafka/pull/1095> to
> > include
> > > > this change.
> > > >
> > > > The change as described on the wiki is:
> > > >
> > > > > The behavior of the replicas and isr arrays will be changed in
> order
> > to
> > > > > support the admin tools, and better represent the state of the
> > cluster:
> > > > >
> > > > > - In version 0, if a broker is down the replicas and isr array
> > will
> > > > > omit the brokers entry and add a REPLICA_NOT_AVAILABLE error
> code.
> > > > > - In version 1, no error code will be set and a the broker id
> will
> > > be
> > > > > included in the replicas and isr array.
> > > > > - Note: A user can still detect if the replica is not
> > available,
> > > by
> > > > > checking if the broker is in the returned broker list.
> > > > >
> > > > >
> > > >
> > > > Being optimistic that this doesn't require to much discussion, I
> would
> > > like
> > > > to re-start the voting process on this thread. If more discussion is
> > > > needed, please don't hesitate to bring it up here.
> > > >
> > > > Ismael, Gwen, Guozhang could you please review and revote based on
> the
> > > > changes.
> > > >
> > > > Thank you,
> > > > Grant
> > > >
> > > > On Sat, Apr 9, 2016 at 1:03 PM, Guozhang Wang <wa...@gmail.com>
> > > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > On Fri, Apr 8, 2016 at 4:36 PM, Gwen Shapira <gw...@confluent.io>
> > > wrote:
> > > > >
> > > > > > +1
> > > > > >
> > > > > > On Fri, Apr 8, 2016 at 2:41 PM, Grant Henke <ghenke@cloudera.com
> >
> > > > wrote:
> > > > > >
> > > > > > > I would like to re-initiate the voting process for the "KIP-4
> > > > Metadata
> > > > > > > Schema changes". This is not a vote for all of KIP-4, but
> > > > specifically
> > > > > > for
> > > > > > > the metadata changes. I have included the exact changes below
> for
> > > > > > clarity:
> > > > > > > >
> > > > > > > > Metadata Request (version 1)
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > MetadataRequest => [topics]
> > > > > > > >
> > > > > > > > Stays the same as version 0 however behavior changes.
> > > > > > > > In version 0 there was no way to request no topics, and and
> > empty
> > > > > list
> > > > > > > > signified all topics.
> > > > > > > > In version 1 a null topics list (size -1 on the wire) will
> > > indicate
> > > > > > that
> > > > > > > a
> > > > > > > > user wants *ALL* topic metadata. Compared to an empty list
> > (size
> > > 0)
> > > > > > which
> > > > > > > > indicates metadata for *NO* topics should be returned.
> > > > > > > > Metadata Response (version 1)
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > MetadataResponse => [brokers] controllerId [topic_metadata]
> > > > > > > > brokers => node_id host port rack
> > > > > > > > node_id => INT32
> > > > > > > > host => STRING
> > > > > > > > port => INT32
> > > > > > > > rack => NULLABLE_STRING
> > > > > > > > controllerId => INT32
> > > > > > > > topic_metadata => topic_error_code topic is_internal
> > > > > > > [partition_metadata]
> > > > > > > > topic_error_code => INT16
> > > > > > > > topic => STRING
> > > > > > > > is_internal => BOOLEAN
> > > > > > > > partition_metadata => partition_error_code partition_id
> > > leader
> > > > > > > [replicas] [isr]
> > > > > > > > partition_error_code => INT16
> > > > > > > > partition_id => INT32
> > > > > > > > leader => INT32
> > > > > > > > replicas => INT32
> > > > > > > > isr => INT32
> > > > > > > >
> > > > > > > > Adds rack, controller_id, and is_internal to the version 0
> > > > response.
> > > > > > > >
> > > > > > >
> > > > > > > The KIP is available here for reference (linked to the Metadata
> > > > schema
> > > > > > > section):
> > > > > > > *
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > > > > >*
> > > > > > >
> > > > > > > A pull request is available implementing the proposed changes
> > here:
> > > > > > > https://github.com/apache/kafka/pull/1095
> > > > > > >
> > > > > > > Here are some links to past discussions on the mailing list:
> > > > > > >
> > > > >
> > >
> http://search-hadoop.com/m/uyzND1pd4T52H1m0u1&subj=Re+KIP+4+Wiki+Update
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1J2IXeSNXAT&subj=Metadata+and+ACLs+wire+protocol+review+KIP+4+
> > > > > > >
> > > > > > > Here is the previous vote discussion (please take a look and
> > > discuss
> > > > > > > there):
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> > > > > > >
> > > > > > > Thank you,
> > > > > > > Grant
> > > > > > > --
> > > > > > > Grant Henke
> > > > > > > Software Engineer | Cloudera
> > > > > > > grant@cloudera.com | twitter.com/gchenke |
> > > > linkedin.com/in/granthenke
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > -- Guozhang
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> >
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [VOTE] KIP-4 Metadata Schema (Round 2)
Posted by Jason Gustafson <ja...@confluent.io>.
+1 (non-binding)
On Mon, Apr 11, 2016 at 6:21 PM, Ismael Juma <is...@juma.me.uk> wrote:
> +1 (non-binding)
>
> Ismael
>
> On Tue, Apr 12, 2016 at 2:19 AM, Jun Rao <ju...@confluent.io> wrote:
>
> > Grant,
> >
> > Thanks for the updated version. +1 from me.
> >
> > Jun
> >
> > On Mon, Apr 11, 2016 at 10:42 AM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> > > Based on the discussion in the previous vote thread
> > > <
> > >
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> > > >
> > > I also would like to include a behavior change to the
> MetadataResponse. I
> > > have update the wiki
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > >
> > > and pull request <https://github.com/apache/kafka/pull/1095> to
> include
> > > this change.
> > >
> > > The change as described on the wiki is:
> > >
> > > > The behavior of the replicas and isr arrays will be changed in order
> to
> > > > support the admin tools, and better represent the state of the
> cluster:
> > > >
> > > > - In version 0, if a broker is down the replicas and isr array
> will
> > > > omit the brokers entry and add a REPLICA_NOT_AVAILABLE error code.
> > > > - In version 1, no error code will be set and a the broker id will
> > be
> > > > included in the replicas and isr array.
> > > > - Note: A user can still detect if the replica is not
> available,
> > by
> > > > checking if the broker is in the returned broker list.
> > > >
> > > >
> > >
> > > Being optimistic that this doesn't require to much discussion, I would
> > like
> > > to re-start the voting process on this thread. If more discussion is
> > > needed, please don't hesitate to bring it up here.
> > >
> > > Ismael, Gwen, Guozhang could you please review and revote based on the
> > > changes.
> > >
> > > Thank you,
> > > Grant
> > >
> > > On Sat, Apr 9, 2016 at 1:03 PM, Guozhang Wang <wa...@gmail.com>
> > wrote:
> > >
> > > > +1
> > > >
> > > > On Fri, Apr 8, 2016 at 4:36 PM, Gwen Shapira <gw...@confluent.io>
> > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > On Fri, Apr 8, 2016 at 2:41 PM, Grant Henke <gh...@cloudera.com>
> > > wrote:
> > > > >
> > > > > > I would like to re-initiate the voting process for the "KIP-4
> > > Metadata
> > > > > > Schema changes". This is not a vote for all of KIP-4, but
> > > specifically
> > > > > for
> > > > > > the metadata changes. I have included the exact changes below for
> > > > > clarity:
> > > > > > >
> > > > > > > Metadata Request (version 1)
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > MetadataRequest => [topics]
> > > > > > >
> > > > > > > Stays the same as version 0 however behavior changes.
> > > > > > > In version 0 there was no way to request no topics, and and
> empty
> > > > list
> > > > > > > signified all topics.
> > > > > > > In version 1 a null topics list (size -1 on the wire) will
> > indicate
> > > > > that
> > > > > > a
> > > > > > > user wants *ALL* topic metadata. Compared to an empty list
> (size
> > 0)
> > > > > which
> > > > > > > indicates metadata for *NO* topics should be returned.
> > > > > > > Metadata Response (version 1)
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > MetadataResponse => [brokers] controllerId [topic_metadata]
> > > > > > > brokers => node_id host port rack
> > > > > > > node_id => INT32
> > > > > > > host => STRING
> > > > > > > port => INT32
> > > > > > > rack => NULLABLE_STRING
> > > > > > > controllerId => INT32
> > > > > > > topic_metadata => topic_error_code topic is_internal
> > > > > > [partition_metadata]
> > > > > > > topic_error_code => INT16
> > > > > > > topic => STRING
> > > > > > > is_internal => BOOLEAN
> > > > > > > partition_metadata => partition_error_code partition_id
> > leader
> > > > > > [replicas] [isr]
> > > > > > > partition_error_code => INT16
> > > > > > > partition_id => INT32
> > > > > > > leader => INT32
> > > > > > > replicas => INT32
> > > > > > > isr => INT32
> > > > > > >
> > > > > > > Adds rack, controller_id, and is_internal to the version 0
> > > response.
> > > > > > >
> > > > > >
> > > > > > The KIP is available here for reference (linked to the Metadata
> > > schema
> > > > > > section):
> > > > > > *
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > > > >*
> > > > > >
> > > > > > A pull request is available implementing the proposed changes
> here:
> > > > > > https://github.com/apache/kafka/pull/1095
> > > > > >
> > > > > > Here are some links to past discussions on the mailing list:
> > > > > >
> > > >
> > http://search-hadoop.com/m/uyzND1pd4T52H1m0u1&subj=Re+KIP+4+Wiki+Update
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1J2IXeSNXAT&subj=Metadata+and+ACLs+wire+protocol+review+KIP+4+
> > > > > >
> > > > > > Here is the previous vote discussion (please take a look and
> > discuss
> > > > > > there):
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> > > > > >
> > > > > > Thank you,
> > > > > > Grant
> > > > > > --
> > > > > > Grant Henke
> > > > > > Software Engineer | Cloudera
> > > > > > grant@cloudera.com | twitter.com/gchenke |
> > > linkedin.com/in/granthenke
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > -- Guozhang
> > > >
> > >
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>
Re: [VOTE] KIP-4 Metadata Schema (Round 2)
Posted by Ismael Juma <is...@juma.me.uk>.
+1 (non-binding)
Ismael
On Tue, Apr 12, 2016 at 2:19 AM, Jun Rao <ju...@confluent.io> wrote:
> Grant,
>
> Thanks for the updated version. +1 from me.
>
> Jun
>
> On Mon, Apr 11, 2016 at 10:42 AM, Grant Henke <gh...@cloudera.com> wrote:
>
> > Based on the discussion in the previous vote thread
> > <
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> > >
> > I also would like to include a behavior change to the MetadataResponse. I
> > have update the wiki
> > <
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > >
> > and pull request <https://github.com/apache/kafka/pull/1095> to include
> > this change.
> >
> > The change as described on the wiki is:
> >
> > > The behavior of the replicas and isr arrays will be changed in order to
> > > support the admin tools, and better represent the state of the cluster:
> > >
> > > - In version 0, if a broker is down the replicas and isr array will
> > > omit the brokers entry and add a REPLICA_NOT_AVAILABLE error code.
> > > - In version 1, no error code will be set and a the broker id will
> be
> > > included in the replicas and isr array.
> > > - Note: A user can still detect if the replica is not available,
> by
> > > checking if the broker is in the returned broker list.
> > >
> > >
> >
> > Being optimistic that this doesn't require to much discussion, I would
> like
> > to re-start the voting process on this thread. If more discussion is
> > needed, please don't hesitate to bring it up here.
> >
> > Ismael, Gwen, Guozhang could you please review and revote based on the
> > changes.
> >
> > Thank you,
> > Grant
> >
> > On Sat, Apr 9, 2016 at 1:03 PM, Guozhang Wang <wa...@gmail.com>
> wrote:
> >
> > > +1
> > >
> > > On Fri, Apr 8, 2016 at 4:36 PM, Gwen Shapira <gw...@confluent.io>
> wrote:
> > >
> > > > +1
> > > >
> > > > On Fri, Apr 8, 2016 at 2:41 PM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > > >
> > > > > I would like to re-initiate the voting process for the "KIP-4
> > Metadata
> > > > > Schema changes". This is not a vote for all of KIP-4, but
> > specifically
> > > > for
> > > > > the metadata changes. I have included the exact changes below for
> > > > clarity:
> > > > > >
> > > > > > Metadata Request (version 1)
> > > > > >
> > > > > >
> > > > > >
> > > > > > MetadataRequest => [topics]
> > > > > >
> > > > > > Stays the same as version 0 however behavior changes.
> > > > > > In version 0 there was no way to request no topics, and and empty
> > > list
> > > > > > signified all topics.
> > > > > > In version 1 a null topics list (size -1 on the wire) will
> indicate
> > > > that
> > > > > a
> > > > > > user wants *ALL* topic metadata. Compared to an empty list (size
> 0)
> > > > which
> > > > > > indicates metadata for *NO* topics should be returned.
> > > > > > Metadata Response (version 1)
> > > > > >
> > > > > >
> > > > > >
> > > > > > MetadataResponse => [brokers] controllerId [topic_metadata]
> > > > > > brokers => node_id host port rack
> > > > > > node_id => INT32
> > > > > > host => STRING
> > > > > > port => INT32
> > > > > > rack => NULLABLE_STRING
> > > > > > controllerId => INT32
> > > > > > topic_metadata => topic_error_code topic is_internal
> > > > > [partition_metadata]
> > > > > > topic_error_code => INT16
> > > > > > topic => STRING
> > > > > > is_internal => BOOLEAN
> > > > > > partition_metadata => partition_error_code partition_id
> leader
> > > > > [replicas] [isr]
> > > > > > partition_error_code => INT16
> > > > > > partition_id => INT32
> > > > > > leader => INT32
> > > > > > replicas => INT32
> > > > > > isr => INT32
> > > > > >
> > > > > > Adds rack, controller_id, and is_internal to the version 0
> > response.
> > > > > >
> > > > >
> > > > > The KIP is available here for reference (linked to the Metadata
> > schema
> > > > > section):
> > > > > *
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > > <
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > > >*
> > > > >
> > > > > A pull request is available implementing the proposed changes here:
> > > > > https://github.com/apache/kafka/pull/1095
> > > > >
> > > > > Here are some links to past discussions on the mailing list:
> > > > >
> > >
> http://search-hadoop.com/m/uyzND1pd4T52H1m0u1&subj=Re+KIP+4+Wiki+Update
> > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1J2IXeSNXAT&subj=Metadata+and+ACLs+wire+protocol+review+KIP+4+
> > > > >
> > > > > Here is the previous vote discussion (please take a look and
> discuss
> > > > > there):
> > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> > > > >
> > > > > Thank you,
> > > > > Grant
> > > > > --
> > > > > Grant Henke
> > > > > Software Engineer | Cloudera
> > > > > grant@cloudera.com | twitter.com/gchenke |
> > linkedin.com/in/granthenke
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
Re: [VOTE] KIP-4 Metadata Schema (Round 2)
Posted by Jun Rao <ju...@confluent.io>.
Grant,
Thanks for the updated version. +1 from me.
Jun
On Mon, Apr 11, 2016 at 10:42 AM, Grant Henke <gh...@cloudera.com> wrote:
> Based on the discussion in the previous vote thread
> <
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> >
> I also would like to include a behavior change to the MetadataResponse. I
> have update the wiki
> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> >
> and pull request <https://github.com/apache/kafka/pull/1095> to include
> this change.
>
> The change as described on the wiki is:
>
> > The behavior of the replicas and isr arrays will be changed in order to
> > support the admin tools, and better represent the state of the cluster:
> >
> > - In version 0, if a broker is down the replicas and isr array will
> > omit the brokers entry and add a REPLICA_NOT_AVAILABLE error code.
> > - In version 1, no error code will be set and a the broker id will be
> > included in the replicas and isr array.
> > - Note: A user can still detect if the replica is not available, by
> > checking if the broker is in the returned broker list.
> >
> >
>
> Being optimistic that this doesn't require to much discussion, I would like
> to re-start the voting process on this thread. If more discussion is
> needed, please don't hesitate to bring it up here.
>
> Ismael, Gwen, Guozhang could you please review and revote based on the
> changes.
>
> Thank you,
> Grant
>
> On Sat, Apr 9, 2016 at 1:03 PM, Guozhang Wang <wa...@gmail.com> wrote:
>
> > +1
> >
> > On Fri, Apr 8, 2016 at 4:36 PM, Gwen Shapira <gw...@confluent.io> wrote:
> >
> > > +1
> > >
> > > On Fri, Apr 8, 2016 at 2:41 PM, Grant Henke <gh...@cloudera.com>
> wrote:
> > >
> > > > I would like to re-initiate the voting process for the "KIP-4
> Metadata
> > > > Schema changes". This is not a vote for all of KIP-4, but
> specifically
> > > for
> > > > the metadata changes. I have included the exact changes below for
> > > clarity:
> > > > >
> > > > > Metadata Request (version 1)
> > > > >
> > > > >
> > > > >
> > > > > MetadataRequest => [topics]
> > > > >
> > > > > Stays the same as version 0 however behavior changes.
> > > > > In version 0 there was no way to request no topics, and and empty
> > list
> > > > > signified all topics.
> > > > > In version 1 a null topics list (size -1 on the wire) will indicate
> > > that
> > > > a
> > > > > user wants *ALL* topic metadata. Compared to an empty list (size 0)
> > > which
> > > > > indicates metadata for *NO* topics should be returned.
> > > > > Metadata Response (version 1)
> > > > >
> > > > >
> > > > >
> > > > > MetadataResponse => [brokers] controllerId [topic_metadata]
> > > > > brokers => node_id host port rack
> > > > > node_id => INT32
> > > > > host => STRING
> > > > > port => INT32
> > > > > rack => NULLABLE_STRING
> > > > > controllerId => INT32
> > > > > topic_metadata => topic_error_code topic is_internal
> > > > [partition_metadata]
> > > > > topic_error_code => INT16
> > > > > topic => STRING
> > > > > is_internal => BOOLEAN
> > > > > partition_metadata => partition_error_code partition_id leader
> > > > [replicas] [isr]
> > > > > partition_error_code => INT16
> > > > > partition_id => INT32
> > > > > leader => INT32
> > > > > replicas => INT32
> > > > > isr => INT32
> > > > >
> > > > > Adds rack, controller_id, and is_internal to the version 0
> response.
> > > > >
> > > >
> > > > The KIP is available here for reference (linked to the Metadata
> schema
> > > > section):
> > > > *
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > <
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > >*
> > > >
> > > > A pull request is available implementing the proposed changes here:
> > > > https://github.com/apache/kafka/pull/1095
> > > >
> > > > Here are some links to past discussions on the mailing list:
> > > >
> > http://search-hadoop.com/m/uyzND1pd4T52H1m0u1&subj=Re+KIP+4+Wiki+Update
> > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1J2IXeSNXAT&subj=Metadata+and+ACLs+wire+protocol+review+KIP+4+
> > > >
> > > > Here is the previous vote discussion (please take a look and discuss
> > > > there):
> > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> > > >
> > > > Thank you,
> > > > Grant
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [VOTE] KIP-4 Metadata Schema (Round 2)
Posted by Guozhang Wang <wa...@gmail.com>.
+1.
On Mon, Apr 11, 2016 at 10:42 AM, Grant Henke <gh...@cloudera.com> wrote:
> Based on the discussion in the previous vote thread
> <
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> >
> I also would like to include a behavior change to the MetadataResponse. I
> have update the wiki
> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> >
> and pull request <https://github.com/apache/kafka/pull/1095> to include
> this change.
>
> The change as described on the wiki is:
>
> > The behavior of the replicas and isr arrays will be changed in order to
> > support the admin tools, and better represent the state of the cluster:
> >
> > - In version 0, if a broker is down the replicas and isr array will
> > omit the brokers entry and add a REPLICA_NOT_AVAILABLE error code.
> > - In version 1, no error code will be set and a the broker id will be
> > included in the replicas and isr array.
> > - Note: A user can still detect if the replica is not available, by
> > checking if the broker is in the returned broker list.
> >
> >
>
> Being optimistic that this doesn't require to much discussion, I would like
> to re-start the voting process on this thread. If more discussion is
> needed, please don't hesitate to bring it up here.
>
> Ismael, Gwen, Guozhang could you please review and revote based on the
> changes.
>
> Thank you,
> Grant
>
> On Sat, Apr 9, 2016 at 1:03 PM, Guozhang Wang <wa...@gmail.com> wrote:
>
> > +1
> >
> > On Fri, Apr 8, 2016 at 4:36 PM, Gwen Shapira <gw...@confluent.io> wrote:
> >
> > > +1
> > >
> > > On Fri, Apr 8, 2016 at 2:41 PM, Grant Henke <gh...@cloudera.com>
> wrote:
> > >
> > > > I would like to re-initiate the voting process for the "KIP-4
> Metadata
> > > > Schema changes". This is not a vote for all of KIP-4, but
> specifically
> > > for
> > > > the metadata changes. I have included the exact changes below for
> > > clarity:
> > > > >
> > > > > Metadata Request (version 1)
> > > > >
> > > > >
> > > > >
> > > > > MetadataRequest => [topics]
> > > > >
> > > > > Stays the same as version 0 however behavior changes.
> > > > > In version 0 there was no way to request no topics, and and empty
> > list
> > > > > signified all topics.
> > > > > In version 1 a null topics list (size -1 on the wire) will indicate
> > > that
> > > > a
> > > > > user wants *ALL* topic metadata. Compared to an empty list (size 0)
> > > which
> > > > > indicates metadata for *NO* topics should be returned.
> > > > > Metadata Response (version 1)
> > > > >
> > > > >
> > > > >
> > > > > MetadataResponse => [brokers] controllerId [topic_metadata]
> > > > > brokers => node_id host port rack
> > > > > node_id => INT32
> > > > > host => STRING
> > > > > port => INT32
> > > > > rack => NULLABLE_STRING
> > > > > controllerId => INT32
> > > > > topic_metadata => topic_error_code topic is_internal
> > > > [partition_metadata]
> > > > > topic_error_code => INT16
> > > > > topic => STRING
> > > > > is_internal => BOOLEAN
> > > > > partition_metadata => partition_error_code partition_id leader
> > > > [replicas] [isr]
> > > > > partition_error_code => INT16
> > > > > partition_id => INT32
> > > > > leader => INT32
> > > > > replicas => INT32
> > > > > isr => INT32
> > > > >
> > > > > Adds rack, controller_id, and is_internal to the version 0
> response.
> > > > >
> > > >
> > > > The KIP is available here for reference (linked to the Metadata
> schema
> > > > section):
> > > > *
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > <
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > > >*
> > > >
> > > > A pull request is available implementing the proposed changes here:
> > > > https://github.com/apache/kafka/pull/1095
> > > >
> > > > Here are some links to past discussions on the mailing list:
> > > >
> > http://search-hadoop.com/m/uyzND1pd4T52H1m0u1&subj=Re+KIP+4+Wiki+Update
> > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1J2IXeSNXAT&subj=Metadata+and+ACLs+wire+protocol+review+KIP+4+
> > > >
> > > > Here is the previous vote discussion (please take a look and discuss
> > > > there):
> > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> > > >
> > > > Thank you,
> > > > Grant
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
--
-- Guozhang
Re: [VOTE] KIP-4 Metadata Schema (Round 2)
Posted by Dana Powers <da...@gmail.com>.
+1
On Apr 11, 2016 21:55, "Gwen Shapira" <gw...@confluent.io> wrote:
> +1
>
> On Mon, Apr 11, 2016 at 10:42 AM, Grant Henke <gh...@cloudera.com> wrote:
> > Based on the discussion in the previous vote thread
> > <
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> >
> > I also would like to include a behavior change to the MetadataResponse. I
> > have update the wiki
> > <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> >
> > and pull request <https://github.com/apache/kafka/pull/1095> to include
> > this change.
> >
> > The change as described on the wiki is:
> >
> >> The behavior of the replicas and isr arrays will be changed in order to
> >> support the admin tools, and better represent the state of the cluster:
> >>
> >> - In version 0, if a broker is down the replicas and isr array will
> >> omit the brokers entry and add a REPLICA_NOT_AVAILABLE error code.
> >> - In version 1, no error code will be set and a the broker id will be
> >> included in the replicas and isr array.
> >> - Note: A user can still detect if the replica is not available,
> by
> >> checking if the broker is in the returned broker list.
> >>
> >>
> >
> > Being optimistic that this doesn't require to much discussion, I would
> like
> > to re-start the voting process on this thread. If more discussion is
> > needed, please don't hesitate to bring it up here.
> >
> > Ismael, Gwen, Guozhang could you please review and revote based on the
> > changes.
> >
> > Thank you,
> > Grant
> >
> > On Sat, Apr 9, 2016 at 1:03 PM, Guozhang Wang <wa...@gmail.com>
> wrote:
> >
> >> +1
> >>
> >> On Fri, Apr 8, 2016 at 4:36 PM, Gwen Shapira <gw...@confluent.io> wrote:
> >>
> >> > +1
> >> >
> >> > On Fri, Apr 8, 2016 at 2:41 PM, Grant Henke <gh...@cloudera.com>
> wrote:
> >> >
> >> > > I would like to re-initiate the voting process for the "KIP-4
> Metadata
> >> > > Schema changes". This is not a vote for all of KIP-4, but
> specifically
> >> > for
> >> > > the metadata changes. I have included the exact changes below for
> >> > clarity:
> >> > > >
> >> > > > Metadata Request (version 1)
> >> > > >
> >> > > >
> >> > > >
> >> > > > MetadataRequest => [topics]
> >> > > >
> >> > > > Stays the same as version 0 however behavior changes.
> >> > > > In version 0 there was no way to request no topics, and and empty
> >> list
> >> > > > signified all topics.
> >> > > > In version 1 a null topics list (size -1 on the wire) will
> indicate
> >> > that
> >> > > a
> >> > > > user wants *ALL* topic metadata. Compared to an empty list (size
> 0)
> >> > which
> >> > > > indicates metadata for *NO* topics should be returned.
> >> > > > Metadata Response (version 1)
> >> > > >
> >> > > >
> >> > > >
> >> > > > MetadataResponse => [brokers] controllerId [topic_metadata]
> >> > > > brokers => node_id host port rack
> >> > > > node_id => INT32
> >> > > > host => STRING
> >> > > > port => INT32
> >> > > > rack => NULLABLE_STRING
> >> > > > controllerId => INT32
> >> > > > topic_metadata => topic_error_code topic is_internal
> >> > > [partition_metadata]
> >> > > > topic_error_code => INT16
> >> > > > topic => STRING
> >> > > > is_internal => BOOLEAN
> >> > > > partition_metadata => partition_error_code partition_id leader
> >> > > [replicas] [isr]
> >> > > > partition_error_code => INT16
> >> > > > partition_id => INT32
> >> > > > leader => INT32
> >> > > > replicas => INT32
> >> > > > isr => INT32
> >> > > >
> >> > > > Adds rack, controller_id, and is_internal to the version 0
> response.
> >> > > >
> >> > >
> >> > > The KIP is available here for reference (linked to the Metadata
> schema
> >> > > section):
> >> > > *
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> >> > > <
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> >> > > >*
> >> > >
> >> > > A pull request is available implementing the proposed changes here:
> >> > > https://github.com/apache/kafka/pull/1095
> >> > >
> >> > > Here are some links to past discussions on the mailing list:
> >> > >
> >> http://search-hadoop.com/m/uyzND1pd4T52H1m0u1&subj=Re+KIP+4+Wiki+Update
> >> > >
> >> > >
> >> >
> >>
> http://search-hadoop.com/m/uyzND1J2IXeSNXAT&subj=Metadata+and+ACLs+wire+protocol+review+KIP+4+
> >> > >
> >> > > Here is the previous vote discussion (please take a look and discuss
> >> > > there):
> >> > >
> >> > >
> >> >
> >>
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> >> > >
> >> > > Thank you,
> >> > > Grant
> >> > > --
> >> > > Grant Henke
> >> > > Software Engineer | Cloudera
> >> > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> -- Guozhang
> >>
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [VOTE] KIP-4 Metadata Schema (Round 2)
Posted by Gwen Shapira <gw...@confluent.io>.
+1
On Mon, Apr 11, 2016 at 10:42 AM, Grant Henke <gh...@cloudera.com> wrote:
> Based on the discussion in the previous vote thread
> <http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema>
> I also would like to include a behavior change to the MetadataResponse. I
> have update the wiki
> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema>
> and pull request <https://github.com/apache/kafka/pull/1095> to include
> this change.
>
> The change as described on the wiki is:
>
>> The behavior of the replicas and isr arrays will be changed in order to
>> support the admin tools, and better represent the state of the cluster:
>>
>> - In version 0, if a broker is down the replicas and isr array will
>> omit the brokers entry and add a REPLICA_NOT_AVAILABLE error code.
>> - In version 1, no error code will be set and a the broker id will be
>> included in the replicas and isr array.
>> - Note: A user can still detect if the replica is not available, by
>> checking if the broker is in the returned broker list.
>>
>>
>
> Being optimistic that this doesn't require to much discussion, I would like
> to re-start the voting process on this thread. If more discussion is
> needed, please don't hesitate to bring it up here.
>
> Ismael, Gwen, Guozhang could you please review and revote based on the
> changes.
>
> Thank you,
> Grant
>
> On Sat, Apr 9, 2016 at 1:03 PM, Guozhang Wang <wa...@gmail.com> wrote:
>
>> +1
>>
>> On Fri, Apr 8, 2016 at 4:36 PM, Gwen Shapira <gw...@confluent.io> wrote:
>>
>> > +1
>> >
>> > On Fri, Apr 8, 2016 at 2:41 PM, Grant Henke <gh...@cloudera.com> wrote:
>> >
>> > > I would like to re-initiate the voting process for the "KIP-4 Metadata
>> > > Schema changes". This is not a vote for all of KIP-4, but specifically
>> > for
>> > > the metadata changes. I have included the exact changes below for
>> > clarity:
>> > > >
>> > > > Metadata Request (version 1)
>> > > >
>> > > >
>> > > >
>> > > > MetadataRequest => [topics]
>> > > >
>> > > > Stays the same as version 0 however behavior changes.
>> > > > In version 0 there was no way to request no topics, and and empty
>> list
>> > > > signified all topics.
>> > > > In version 1 a null topics list (size -1 on the wire) will indicate
>> > that
>> > > a
>> > > > user wants *ALL* topic metadata. Compared to an empty list (size 0)
>> > which
>> > > > indicates metadata for *NO* topics should be returned.
>> > > > Metadata Response (version 1)
>> > > >
>> > > >
>> > > >
>> > > > MetadataResponse => [brokers] controllerId [topic_metadata]
>> > > > brokers => node_id host port rack
>> > > > node_id => INT32
>> > > > host => STRING
>> > > > port => INT32
>> > > > rack => NULLABLE_STRING
>> > > > controllerId => INT32
>> > > > topic_metadata => topic_error_code topic is_internal
>> > > [partition_metadata]
>> > > > topic_error_code => INT16
>> > > > topic => STRING
>> > > > is_internal => BOOLEAN
>> > > > partition_metadata => partition_error_code partition_id leader
>> > > [replicas] [isr]
>> > > > partition_error_code => INT16
>> > > > partition_id => INT32
>> > > > leader => INT32
>> > > > replicas => INT32
>> > > > isr => INT32
>> > > >
>> > > > Adds rack, controller_id, and is_internal to the version 0 response.
>> > > >
>> > >
>> > > The KIP is available here for reference (linked to the Metadata schema
>> > > section):
>> > > *
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
>> > > <
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
>> > > >*
>> > >
>> > > A pull request is available implementing the proposed changes here:
>> > > https://github.com/apache/kafka/pull/1095
>> > >
>> > > Here are some links to past discussions on the mailing list:
>> > >
>> http://search-hadoop.com/m/uyzND1pd4T52H1m0u1&subj=Re+KIP+4+Wiki+Update
>> > >
>> > >
>> >
>> http://search-hadoop.com/m/uyzND1J2IXeSNXAT&subj=Metadata+and+ACLs+wire+protocol+review+KIP+4+
>> > >
>> > > Here is the previous vote discussion (please take a look and discuss
>> > > there):
>> > >
>> > >
>> >
>> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
>> > >
>> > > Thank you,
>> > > Grant
>> > > --
>> > > Grant Henke
>> > > Software Engineer | Cloudera
>> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>> > >
>> >
>>
>>
>>
>> --
>> -- Guozhang
>>
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [VOTE] KIP-4 Metadata Schema (Round 2)
Posted by Grant Henke <gh...@cloudera.com>.
Based on the discussion in the previous vote thread
<http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema>
I also would like to include a behavior change to the MetadataResponse. I
have update the wiki
<https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema>
and pull request <https://github.com/apache/kafka/pull/1095> to include
this change.
The change as described on the wiki is:
> The behavior of the replicas and isr arrays will be changed in order to
> support the admin tools, and better represent the state of the cluster:
>
> - In version 0, if a broker is down the replicas and isr array will
> omit the brokers entry and add a REPLICA_NOT_AVAILABLE error code.
> - In version 1, no error code will be set and a the broker id will be
> included in the replicas and isr array.
> - Note: A user can still detect if the replica is not available, by
> checking if the broker is in the returned broker list.
>
>
Being optimistic that this doesn't require to much discussion, I would like
to re-start the voting process on this thread. If more discussion is
needed, please don't hesitate to bring it up here.
Ismael, Gwen, Guozhang could you please review and revote based on the
changes.
Thank you,
Grant
On Sat, Apr 9, 2016 at 1:03 PM, Guozhang Wang <wa...@gmail.com> wrote:
> +1
>
> On Fri, Apr 8, 2016 at 4:36 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > +1
> >
> > On Fri, Apr 8, 2016 at 2:41 PM, Grant Henke <gh...@cloudera.com> wrote:
> >
> > > I would like to re-initiate the voting process for the "KIP-4 Metadata
> > > Schema changes". This is not a vote for all of KIP-4, but specifically
> > for
> > > the metadata changes. I have included the exact changes below for
> > clarity:
> > > >
> > > > Metadata Request (version 1)
> > > >
> > > >
> > > >
> > > > MetadataRequest => [topics]
> > > >
> > > > Stays the same as version 0 however behavior changes.
> > > > In version 0 there was no way to request no topics, and and empty
> list
> > > > signified all topics.
> > > > In version 1 a null topics list (size -1 on the wire) will indicate
> > that
> > > a
> > > > user wants *ALL* topic metadata. Compared to an empty list (size 0)
> > which
> > > > indicates metadata for *NO* topics should be returned.
> > > > Metadata Response (version 1)
> > > >
> > > >
> > > >
> > > > MetadataResponse => [brokers] controllerId [topic_metadata]
> > > > brokers => node_id host port rack
> > > > node_id => INT32
> > > > host => STRING
> > > > port => INT32
> > > > rack => NULLABLE_STRING
> > > > controllerId => INT32
> > > > topic_metadata => topic_error_code topic is_internal
> > > [partition_metadata]
> > > > topic_error_code => INT16
> > > > topic => STRING
> > > > is_internal => BOOLEAN
> > > > partition_metadata => partition_error_code partition_id leader
> > > [replicas] [isr]
> > > > partition_error_code => INT16
> > > > partition_id => INT32
> > > > leader => INT32
> > > > replicas => INT32
> > > > isr => INT32
> > > >
> > > > Adds rack, controller_id, and is_internal to the version 0 response.
> > > >
> > >
> > > The KIP is available here for reference (linked to the Metadata schema
> > > section):
> > > *
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > > >*
> > >
> > > A pull request is available implementing the proposed changes here:
> > > https://github.com/apache/kafka/pull/1095
> > >
> > > Here are some links to past discussions on the mailing list:
> > >
> http://search-hadoop.com/m/uyzND1pd4T52H1m0u1&subj=Re+KIP+4+Wiki+Update
> > >
> > >
> >
> http://search-hadoop.com/m/uyzND1J2IXeSNXAT&subj=Metadata+and+ACLs+wire+protocol+review+KIP+4+
> > >
> > > Here is the previous vote discussion (please take a look and discuss
> > > there):
> > >
> > >
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> > >
> > > Thank you,
> > > Grant
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>
>
>
> --
> -- Guozhang
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [VOTE] KIP-4 Metadata Schema (Round 2)
Posted by Guozhang Wang <wa...@gmail.com>.
+1
On Fri, Apr 8, 2016 at 4:36 PM, Gwen Shapira <gw...@confluent.io> wrote:
> +1
>
> On Fri, Apr 8, 2016 at 2:41 PM, Grant Henke <gh...@cloudera.com> wrote:
>
> > I would like to re-initiate the voting process for the "KIP-4 Metadata
> > Schema changes". This is not a vote for all of KIP-4, but specifically
> for
> > the metadata changes. I have included the exact changes below for
> clarity:
> > >
> > > Metadata Request (version 1)
> > >
> > >
> > >
> > > MetadataRequest => [topics]
> > >
> > > Stays the same as version 0 however behavior changes.
> > > In version 0 there was no way to request no topics, and and empty list
> > > signified all topics.
> > > In version 1 a null topics list (size -1 on the wire) will indicate
> that
> > a
> > > user wants *ALL* topic metadata. Compared to an empty list (size 0)
> which
> > > indicates metadata for *NO* topics should be returned.
> > > Metadata Response (version 1)
> > >
> > >
> > >
> > > MetadataResponse => [brokers] controllerId [topic_metadata]
> > > brokers => node_id host port rack
> > > node_id => INT32
> > > host => STRING
> > > port => INT32
> > > rack => NULLABLE_STRING
> > > controllerId => INT32
> > > topic_metadata => topic_error_code topic is_internal
> > [partition_metadata]
> > > topic_error_code => INT16
> > > topic => STRING
> > > is_internal => BOOLEAN
> > > partition_metadata => partition_error_code partition_id leader
> > [replicas] [isr]
> > > partition_error_code => INT16
> > > partition_id => INT32
> > > leader => INT32
> > > replicas => INT32
> > > isr => INT32
> > >
> > > Adds rack, controller_id, and is_internal to the version 0 response.
> > >
> >
> > The KIP is available here for reference (linked to the Metadata schema
> > section):
> > *
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > <
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> > >*
> >
> > A pull request is available implementing the proposed changes here:
> > https://github.com/apache/kafka/pull/1095
> >
> > Here are some links to past discussions on the mailing list:
> > http://search-hadoop.com/m/uyzND1pd4T52H1m0u1&subj=Re+KIP+4+Wiki+Update
> >
> >
> http://search-hadoop.com/m/uyzND1J2IXeSNXAT&subj=Metadata+and+ACLs+wire+protocol+review+KIP+4+
> >
> > Here is the previous vote discussion (please take a look and discuss
> > there):
> >
> >
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
> >
> > Thank you,
> > Grant
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
--
-- Guozhang
Re: [VOTE] KIP-4 Metadata Schema (Round 2)
Posted by Gwen Shapira <gw...@confluent.io>.
+1
On Fri, Apr 8, 2016 at 2:41 PM, Grant Henke <gh...@cloudera.com> wrote:
> I would like to re-initiate the voting process for the "KIP-4 Metadata
> Schema changes". This is not a vote for all of KIP-4, but specifically for
> the metadata changes. I have included the exact changes below for clarity:
> >
> > Metadata Request (version 1)
> >
> >
> >
> > MetadataRequest => [topics]
> >
> > Stays the same as version 0 however behavior changes.
> > In version 0 there was no way to request no topics, and and empty list
> > signified all topics.
> > In version 1 a null topics list (size -1 on the wire) will indicate that
> a
> > user wants *ALL* topic metadata. Compared to an empty list (size 0) which
> > indicates metadata for *NO* topics should be returned.
> > Metadata Response (version 1)
> >
> >
> >
> > MetadataResponse => [brokers] controllerId [topic_metadata]
> > brokers => node_id host port rack
> > node_id => INT32
> > host => STRING
> > port => INT32
> > rack => NULLABLE_STRING
> > controllerId => INT32
> > topic_metadata => topic_error_code topic is_internal
> [partition_metadata]
> > topic_error_code => INT16
> > topic => STRING
> > is_internal => BOOLEAN
> > partition_metadata => partition_error_code partition_id leader
> [replicas] [isr]
> > partition_error_code => INT16
> > partition_id => INT32
> > leader => INT32
> > replicas => INT32
> > isr => INT32
> >
> > Adds rack, controller_id, and is_internal to the version 0 response.
> >
>
> The KIP is available here for reference (linked to the Metadata schema
> section):
> *
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-MetadataSchema
> >*
>
> A pull request is available implementing the proposed changes here:
> https://github.com/apache/kafka/pull/1095
>
> Here are some links to past discussions on the mailing list:
> http://search-hadoop.com/m/uyzND1pd4T52H1m0u1&subj=Re+KIP+4+Wiki+Update
>
> http://search-hadoop.com/m/uyzND1J2IXeSNXAT&subj=Metadata+and+ACLs+wire+protocol+review+KIP+4+
>
> Here is the previous vote discussion (please take a look and discuss
> there):
>
> http://search-hadoop.com/m/uyzND1xlaiU10QlYX&subj=+VOTE+KIP+4+Metadata+Schema
>
> Thank you,
> Grant
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>