You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Calvin Chen <pi...@hotmail.com> on 2019/03/08 07:27:10 UTC
question about kafka topic
Hi,
I have a question about kafka topic, recently we face kafka client sending message to kafka topic issue, got error about offset, and client can not send message to kafka cluster.
My question is, since we configure kafka servers to one cluster, when cluster get message, will it try it best to deliver message to good host's partition? say, if cluster forward message to host A's topic partition, but it failed, then will cluster find another host's partition and redeliver the message? or just simply return error message to client?
Is it configurable? what is the parameter to configure it?
Thanks
-Calvin
Re: question about kafka topic
Posted by "Matthias J. Sax" <ma...@confluent.io>.
What error do you exactly see?
On 3/10/19 10:35 AM, K R, Rakesh wrote:
> Sorry I did not mention one point. I am seeing this error on consumer side.
>
> -----Original Message-----
> From: Matthias J. Sax [mailto:matthias@confluent.io]
> Sent: Sunday, March 10, 2019 11:01 PM
> To: users@kafka.apache.org
> Subject: Re: question about kafka topic
>
> Writes, must always go to the partition leader, ie, if an error occurs, the message cannot simply be delivered to a different broker.
>
> However, if a broker is in bad shape, it could get leaderhship revoked and a healthy broker would get leadership assigned. The producer can detect this automatically.
>
> If you don't want to get an error back but you want the producer to retry, you can increase config parameter `retries`. Note, that there are two types of errors, retryable and non-retryable. For non-retryable errors, the producer will always rethrow the error, because, as the name suggests, it won't make sense to retry because the same error would be returned by the broker on retries, too.
>
> Hope this helps.
>
>
> -Matthias
>
>
>
> On 3/7/19 11:27 PM, Calvin Chen wrote:
>> Hi,
>> I have a question about kafka topic, recently we face kafka client sending message to kafka topic issue, got error about offset, and client can not send message to kafka cluster.
>>
>> My question is, since we configure kafka servers to one cluster, when cluster get message, will it try it best to deliver message to good host's partition? say, if cluster forward message to host A's topic partition, but it failed, then will cluster find another host's partition and redeliver the message? or just simply return error message to client?
>>
>> Is it configurable? what is the parameter to configure it?
>>
>> Thanks
>> -Calvin
>>
>
RE: question about kafka topic
Posted by "K R, Rakesh" <rk...@hpe.com>.
Sorry I did not mention one point. I am seeing this error on consumer side.
-----Original Message-----
From: Matthias J. Sax [mailto:matthias@confluent.io]
Sent: Sunday, March 10, 2019 11:01 PM
To: users@kafka.apache.org
Subject: Re: question about kafka topic
Writes, must always go to the partition leader, ie, if an error occurs, the message cannot simply be delivered to a different broker.
However, if a broker is in bad shape, it could get leaderhship revoked and a healthy broker would get leadership assigned. The producer can detect this automatically.
If you don't want to get an error back but you want the producer to retry, you can increase config parameter `retries`. Note, that there are two types of errors, retryable and non-retryable. For non-retryable errors, the producer will always rethrow the error, because, as the name suggests, it won't make sense to retry because the same error would be returned by the broker on retries, too.
Hope this helps.
-Matthias
On 3/7/19 11:27 PM, Calvin Chen wrote:
> Hi,
> I have a question about kafka topic, recently we face kafka client sending message to kafka topic issue, got error about offset, and client can not send message to kafka cluster.
>
> My question is, since we configure kafka servers to one cluster, when cluster get message, will it try it best to deliver message to good host's partition? say, if cluster forward message to host A's topic partition, but it failed, then will cluster find another host's partition and redeliver the message? or just simply return error message to client?
>
> Is it configurable? what is the parameter to configure it?
>
> Thanks
> -Calvin
>
Re: question about kafka topic
Posted by "Matthias J. Sax" <ma...@confluent.io>.
Writes, must always go to the partition leader, ie, if an error occurs,
the message cannot simply be delivered to a different broker.
However, if a broker is in bad shape, it could get leaderhship revoked
and a healthy broker would get leadership assigned. The producer can
detect this automatically.
If you don't want to get an error back but you want the producer to
retry, you can increase config parameter `retries`. Note, that there are
two types of errors, retryable and non-retryable. For non-retryable
errors, the producer will always rethrow the error, because, as the name
suggests, it won't make sense to retry because the same error would be
returned by the broker on retries, too.
Hope this helps.
-Matthias
On 3/7/19 11:27 PM, Calvin Chen wrote:
> Hi,
> I have a question about kafka topic, recently we face kafka client sending message to kafka topic issue, got error about offset, and client can not send message to kafka cluster.
>
> My question is, since we configure kafka servers to one cluster, when cluster get message, will it try it best to deliver message to good host's partition? say, if cluster forward message to host A's topic partition, but it failed, then will cluster find another host's partition and redeliver the message? or just simply return error message to client?
>
> Is it configurable? what is the parameter to configure it?
>
> Thanks
> -Calvin
>