You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Evan Huus <ev...@shopify.com> on 2014/11/25 21:29:17 UTC
MetadataResponse error code handling
Hi folks,
I was wondering in which cases the PartitionMetadata section of a
MetadataResponse [1] can contain useful information?
I had been working under the assumption that when the PartitionErrorCode
was 0 (NoError) then the rest of the data would be legitimate, and that
when the error was non-0 then the rest of the data would be blank/empty/etc.
However, I recently discovered a JIRA ticket [2] which suggests otherwise,
namely that there are certain error codes for which parts of the remainder
of the metadata may still be filled in. I have not found any explicit
documentation for which error codes this is true. Does anybody know?
Thanks,
Evan
P.S. I'm not 100% sure if this kind of question should go on users@ or dev@.
Please let me know if I'm in the wrong place.
[1]
https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-MetadataResponse
[2] https://issues.apache.org/jira/browse/KAFKA-1609
Re: MetadataResponse error code handling
Posted by Evan Huus <ev...@shopify.com>.
Thanks, that's exactly what I needed to know.
Evan
On Wed, Nov 26, 2014 at 2:22 PM, Guozhang Wang <wa...@gmail.com> wrote:
> For the new producer, the only error code that will result in invalid
> partition metadata is LeaderNotAvailable, and as for the implementation
> details of the KAFKA-1609 patch, it handles this error code by setting the
> leader field to null.
>
> Guozhang
>
> On Wed, Nov 26, 2014 at 11:08 AM, Evan Huus <ev...@shopify.com> wrote:
>
> > Hi Guozhang, thanks for the information.
> >
> > I am implementing/maintaining the Golang producer [1] so I do need to
> worry
> > about these details.
> >
> > You mention LeaderNotAvailable as the only one, but the JIRA ticket I
> > linked also talks about ReplicaNotAvailable. Are both of these ignored?
> Or
> > is it that LeaderNotAvailable causes the metadata to be ignored, and
> *all*
> > other partition-specific errors (including ReplicaNotAvailable) still
> carry
> > legitimate data?
> >
> > Thanks,
> > Evan
> >
> > [1] https://github.com/Shopify/sarama
> >
> > On Wed, Nov 26, 2014 at 1:11 PM, Guozhang Wang <wa...@gmail.com>
> wrote:
> >
> > > Hi Evan,
> > >
> > > The error code is per-partition, and for new producer only
> > > LeaderNotAvailable error will cause it to ignore the metadata for that
> > > partition. If you are using the producer clients directly you would not
> > > need to worry about these error handling details; if you are fetching
> the
> > > metadata using a metadata request yourself and then you need to handle
> > the
> > > error code accordingly.
> > >
> > > I agree that such documentation is kind of lacking for now, and we
> would
> > > better add that in the response class Java doc.
> > >
> > > Guozhang
> > >
> > >
> > > On Tue, Nov 25, 2014 at 12:29 PM, Evan Huus <ev...@shopify.com>
> > wrote:
> > >
> > > > Hi folks,
> > > >
> > > > I was wondering in which cases the PartitionMetadata section of a
> > > > MetadataResponse [1] can contain useful information?
> > > >
> > > > I had been working under the assumption that when the
> > PartitionErrorCode
> > > > was 0 (NoError) then the rest of the data would be legitimate, and
> that
> > > > when the error was non-0 then the rest of the data would be
> > > > blank/empty/etc.
> > > >
> > > > However, I recently discovered a JIRA ticket [2] which suggests
> > > otherwise,
> > > > namely that there are certain error codes for which parts of the
> > > remainder
> > > > of the metadata may still be filled in. I have not found any explicit
> > > > documentation for which error codes this is true. Does anybody know?
> > > >
> > > > Thanks,
> > > > Evan
> > > >
> > > > P.S. I'm not 100% sure if this kind of question should go on users@
> or
> > > > dev@.
> > > > Please let me know if I'm in the wrong place.
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-MetadataResponse
> > > > [2] https://issues.apache.org/jira/browse/KAFKA-1609
> > > >
> > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
>
>
>
> --
> -- Guozhang
>
Re: MetadataResponse error code handling
Posted by Guozhang Wang <wa...@gmail.com>.
For the new producer, the only error code that will result in invalid
partition metadata is LeaderNotAvailable, and as for the implementation
details of the KAFKA-1609 patch, it handles this error code by setting the
leader field to null.
Guozhang
On Wed, Nov 26, 2014 at 11:08 AM, Evan Huus <ev...@shopify.com> wrote:
> Hi Guozhang, thanks for the information.
>
> I am implementing/maintaining the Golang producer [1] so I do need to worry
> about these details.
>
> You mention LeaderNotAvailable as the only one, but the JIRA ticket I
> linked also talks about ReplicaNotAvailable. Are both of these ignored? Or
> is it that LeaderNotAvailable causes the metadata to be ignored, and *all*
> other partition-specific errors (including ReplicaNotAvailable) still carry
> legitimate data?
>
> Thanks,
> Evan
>
> [1] https://github.com/Shopify/sarama
>
> On Wed, Nov 26, 2014 at 1:11 PM, Guozhang Wang <wa...@gmail.com> wrote:
>
> > Hi Evan,
> >
> > The error code is per-partition, and for new producer only
> > LeaderNotAvailable error will cause it to ignore the metadata for that
> > partition. If you are using the producer clients directly you would not
> > need to worry about these error handling details; if you are fetching the
> > metadata using a metadata request yourself and then you need to handle
> the
> > error code accordingly.
> >
> > I agree that such documentation is kind of lacking for now, and we would
> > better add that in the response class Java doc.
> >
> > Guozhang
> >
> >
> > On Tue, Nov 25, 2014 at 12:29 PM, Evan Huus <ev...@shopify.com>
> wrote:
> >
> > > Hi folks,
> > >
> > > I was wondering in which cases the PartitionMetadata section of a
> > > MetadataResponse [1] can contain useful information?
> > >
> > > I had been working under the assumption that when the
> PartitionErrorCode
> > > was 0 (NoError) then the rest of the data would be legitimate, and that
> > > when the error was non-0 then the rest of the data would be
> > > blank/empty/etc.
> > >
> > > However, I recently discovered a JIRA ticket [2] which suggests
> > otherwise,
> > > namely that there are certain error codes for which parts of the
> > remainder
> > > of the metadata may still be filled in. I have not found any explicit
> > > documentation for which error codes this is true. Does anybody know?
> > >
> > > Thanks,
> > > Evan
> > >
> > > P.S. I'm not 100% sure if this kind of question should go on users@ or
> > > dev@.
> > > Please let me know if I'm in the wrong place.
> > >
> > > [1]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-MetadataResponse
> > > [2] https://issues.apache.org/jira/browse/KAFKA-1609
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
--
-- Guozhang
Re: MetadataResponse error code handling
Posted by Evan Huus <ev...@shopify.com>.
Hi Guozhang, thanks for the information.
I am implementing/maintaining the Golang producer [1] so I do need to worry
about these details.
You mention LeaderNotAvailable as the only one, but the JIRA ticket I
linked also talks about ReplicaNotAvailable. Are both of these ignored? Or
is it that LeaderNotAvailable causes the metadata to be ignored, and *all*
other partition-specific errors (including ReplicaNotAvailable) still carry
legitimate data?
Thanks,
Evan
[1] https://github.com/Shopify/sarama
On Wed, Nov 26, 2014 at 1:11 PM, Guozhang Wang <wa...@gmail.com> wrote:
> Hi Evan,
>
> The error code is per-partition, and for new producer only
> LeaderNotAvailable error will cause it to ignore the metadata for that
> partition. If you are using the producer clients directly you would not
> need to worry about these error handling details; if you are fetching the
> metadata using a metadata request yourself and then you need to handle the
> error code accordingly.
>
> I agree that such documentation is kind of lacking for now, and we would
> better add that in the response class Java doc.
>
> Guozhang
>
>
> On Tue, Nov 25, 2014 at 12:29 PM, Evan Huus <ev...@shopify.com> wrote:
>
> > Hi folks,
> >
> > I was wondering in which cases the PartitionMetadata section of a
> > MetadataResponse [1] can contain useful information?
> >
> > I had been working under the assumption that when the PartitionErrorCode
> > was 0 (NoError) then the rest of the data would be legitimate, and that
> > when the error was non-0 then the rest of the data would be
> > blank/empty/etc.
> >
> > However, I recently discovered a JIRA ticket [2] which suggests
> otherwise,
> > namely that there are certain error codes for which parts of the
> remainder
> > of the metadata may still be filled in. I have not found any explicit
> > documentation for which error codes this is true. Does anybody know?
> >
> > Thanks,
> > Evan
> >
> > P.S. I'm not 100% sure if this kind of question should go on users@ or
> > dev@.
> > Please let me know if I'm in the wrong place.
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-MetadataResponse
> > [2] https://issues.apache.org/jira/browse/KAFKA-1609
> >
>
>
>
> --
> -- Guozhang
>
Re: MetadataResponse error code handling
Posted by Guozhang Wang <wa...@gmail.com>.
Hi Evan,
The error code is per-partition, and for new producer only
LeaderNotAvailable error will cause it to ignore the metadata for that
partition. If you are using the producer clients directly you would not
need to worry about these error handling details; if you are fetching the
metadata using a metadata request yourself and then you need to handle the
error code accordingly.
I agree that such documentation is kind of lacking for now, and we would
better add that in the response class Java doc.
Guozhang
On Tue, Nov 25, 2014 at 12:29 PM, Evan Huus <ev...@shopify.com> wrote:
> Hi folks,
>
> I was wondering in which cases the PartitionMetadata section of a
> MetadataResponse [1] can contain useful information?
>
> I had been working under the assumption that when the PartitionErrorCode
> was 0 (NoError) then the rest of the data would be legitimate, and that
> when the error was non-0 then the rest of the data would be
> blank/empty/etc.
>
> However, I recently discovered a JIRA ticket [2] which suggests otherwise,
> namely that there are certain error codes for which parts of the remainder
> of the metadata may still be filled in. I have not found any explicit
> documentation for which error codes this is true. Does anybody know?
>
> Thanks,
> Evan
>
> P.S. I'm not 100% sure if this kind of question should go on users@ or
> dev@.
> Please let me know if I'm in the wrong place.
>
> [1]
>
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-MetadataResponse
> [2] https://issues.apache.org/jira/browse/KAFKA-1609
>
--
-- Guozhang