You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Bhavesh Mistry <mi...@gmail.com> on 2015/01/02 21:37:21 UTC
Follow-up On Important Issues for 0.8.2
Hi Kafka Dev Team,
I am following-up with you guys regarding New (Java) Producer behavior in
event of network or firewall rules. I just wanted to make Java Producer
resilient of any network or firewall issues, and does not become
single-point of failure in application:
1) Jira Issue https://issues.apache.org/jira/browse/KAFKA-1788
https://issues.apache.org/jira/browse/KAFKA-1788?focusedCommentId=14259235&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14259235
What should be the behavior of the Producer when it can not reach leader
broker, but metadata reported broker is leader for that partition (via
other broker) ? Should the record-error-rate be counted and Call Back
should be called with error or not ?
1) *record-error-rate* metric remain zero despite following firewall rule.
In my opinion, it should have called
org.apache.kafka.clients.producer.Callback but I did not see that happening
either one or two are brokers not reachable.
2) Is jira ticket https://issues.apache.org/jira/browse/KAFKA-1788 will
be merged to 0.8.2 ? This will give the ability to close the producer in
event of lost connectivity to broker if io thread misbehave (does not end)
?
Thanks for your help !
Thanks,
Bhavesh
Re: Follow-up On Important Issues for 0.8.2
Posted by Bhavesh Mistry <mi...@gmail.com>.
Hi Kafka Dev Team,
We have to similar change for # 2 issue, I would like to know if /KAFKA-1788
<https://issues.apache.org/jira/browse/KAFKA-1788> will be part of
release. Secondly, issue #1, I would like to know what is expected
behavior of metrics error counting and call back in (async mode) when there
is no issues with ZK or Brokers but network layer (firewalll or DNS issue)
only.
If need to be, I can file a jira ticket after understanding behavior and I
can test expected behavior as well.
Thanks,
Bhavesh
On Fri, Jan 2, 2015 at 12:37 PM, Bhavesh Mistry <mi...@gmail.com>
wrote:
> Hi Kafka Dev Team,
>
> I am following-up with you guys regarding New (Java) Producer behavior in
> event of network or firewall rules. I just wanted to make Java Producer
> resilient of any network or firewall issues, and does not become
> single-point of failure in application:
>
> 1) Jira Issue https://issues.apache.org/jira/browse/KAFKA-1788
>
>
> https://issues.apache.org/jira/browse/KAFKA-1788?focusedCommentId=14259235&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14259235
>
> What should be the behavior of the Producer when it can not reach leader
> broker, but metadata reported broker is leader for that partition (via
> other broker) ? Should the record-error-rate be counted and Call Back
> should be called with error or not ?
>
> 1) *record-error-rate* metric remain zero despite following firewall
> rule. In my opinion, it should have called
> org.apache.kafka.clients.producer.Callback but I did not see that happening
> either one or two are brokers not reachable.
>
> 2) Is jira ticket https://issues.apache.org/jira/browse/KAFKA-1788 will
> be merged to 0.8.2 ? This will give the ability to close the producer in
> event of lost connectivity to broker if io thread misbehave (does not end)
> ?
>
> Thanks for your help !
>
> Thanks,
> Bhavesh
>
Re: Follow-up On Important Issues for 0.8.2
Posted by Ewen Cheslack-Postava <ew...@confluent.io>.
Bhavesh,
1. I would expect the behavior you're suggesting as well. Any fix for the
JIRA should address this, and I raised that point when reviewing the
current version of the patch.
2. It's currently marked for 0.8.3 so it probably won't make it into 0.8.2,
especially if there isn't a patch for it ready soon. I think that's
probably the right choice -- it's a pretty unusual corner case and the
current behavior is annoying, but not a critical bug.
-Ewen
On Thu, Jan 8, 2015 at 10:59 PM, Bhavesh Mistry <mi...@gmail.com>
wrote:
> Adding User Community to see if any one knows behavior of Producer for
> issue #1) and status of 2).
>
>
> Thanks,
>
> Bhavesh
>
> On Fri, Jan 2, 2015 at 12:37 PM, Bhavesh Mistry <
> mistry.p.bhavesh@gmail.com>
> wrote:
>
> > Hi Kafka Dev Team,
> >
> > I am following-up with you guys regarding New (Java) Producer behavior in
> > event of network or firewall rules. I just wanted to make Java Producer
> > resilient of any network or firewall issues, and does not become
> > single-point of failure in application:
> >
> > 1) Jira Issue https://issues.apache.org/jira/browse/KAFKA-1788
> >
> >
> >
> https://issues.apache.org/jira/browse/KAFKA-1788?focusedCommentId=14259235&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14259235
> >
> > What should be the behavior of the Producer when it can not reach leader
> > broker, but metadata reported broker is leader for that partition (via
> > other broker) ? Should the record-error-rate be counted and Call Back
> > should be called with error or not ?
> >
> > 1) *record-error-rate* metric remain zero despite following firewall
> > rule. In my opinion, it should have called
> > org.apache.kafka.clients.producer.Callback but I did not see that
> happening
> > either one or two are brokers not reachable.
> >
> > 2) Is jira ticket https://issues.apache.org/jira/browse/KAFKA-1788
> will
> > be merged to 0.8.2 ? This will give the ability to close the producer in
> > event of lost connectivity to broker if io thread misbehave (does not
> end)
> > ?
> >
> > Thanks for your help !
> >
> > Thanks,
> > Bhavesh
> >
>
--
Thanks,
Ewen
Re: Follow-up On Important Issues for 0.8.2
Posted by Ewen Cheslack-Postava <ew...@confluent.io>.
Bhavesh,
1. I would expect the behavior you're suggesting as well. Any fix for the
JIRA should address this, and I raised that point when reviewing the
current version of the patch.
2. It's currently marked for 0.8.3 so it probably won't make it into 0.8.2,
especially if there isn't a patch for it ready soon. I think that's
probably the right choice -- it's a pretty unusual corner case and the
current behavior is annoying, but not a critical bug.
-Ewen
On Thu, Jan 8, 2015 at 10:59 PM, Bhavesh Mistry <mi...@gmail.com>
wrote:
> Adding User Community to see if any one knows behavior of Producer for
> issue #1) and status of 2).
>
>
> Thanks,
>
> Bhavesh
>
> On Fri, Jan 2, 2015 at 12:37 PM, Bhavesh Mistry <
> mistry.p.bhavesh@gmail.com>
> wrote:
>
> > Hi Kafka Dev Team,
> >
> > I am following-up with you guys regarding New (Java) Producer behavior in
> > event of network or firewall rules. I just wanted to make Java Producer
> > resilient of any network or firewall issues, and does not become
> > single-point of failure in application:
> >
> > 1) Jira Issue https://issues.apache.org/jira/browse/KAFKA-1788
> >
> >
> >
> https://issues.apache.org/jira/browse/KAFKA-1788?focusedCommentId=14259235&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14259235
> >
> > What should be the behavior of the Producer when it can not reach leader
> > broker, but metadata reported broker is leader for that partition (via
> > other broker) ? Should the record-error-rate be counted and Call Back
> > should be called with error or not ?
> >
> > 1) *record-error-rate* metric remain zero despite following firewall
> > rule. In my opinion, it should have called
> > org.apache.kafka.clients.producer.Callback but I did not see that
> happening
> > either one or two are brokers not reachable.
> >
> > 2) Is jira ticket https://issues.apache.org/jira/browse/KAFKA-1788
> will
> > be merged to 0.8.2 ? This will give the ability to close the producer in
> > event of lost connectivity to broker if io thread misbehave (does not
> end)
> > ?
> >
> > Thanks for your help !
> >
> > Thanks,
> > Bhavesh
> >
>
--
Thanks,
Ewen
Re: Follow-up On Important Issues for 0.8.2
Posted by Bhavesh Mistry <mi...@gmail.com>.
Adding User Community to see if any one knows behavior of Producer for
issue #1) and status of 2).
Thanks,
Bhavesh
On Fri, Jan 2, 2015 at 12:37 PM, Bhavesh Mistry <mi...@gmail.com>
wrote:
> Hi Kafka Dev Team,
>
> I am following-up with you guys regarding New (Java) Producer behavior in
> event of network or firewall rules. I just wanted to make Java Producer
> resilient of any network or firewall issues, and does not become
> single-point of failure in application:
>
> 1) Jira Issue https://issues.apache.org/jira/browse/KAFKA-1788
>
>
> https://issues.apache.org/jira/browse/KAFKA-1788?focusedCommentId=14259235&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14259235
>
> What should be the behavior of the Producer when it can not reach leader
> broker, but metadata reported broker is leader for that partition (via
> other broker) ? Should the record-error-rate be counted and Call Back
> should be called with error or not ?
>
> 1) *record-error-rate* metric remain zero despite following firewall
> rule. In my opinion, it should have called
> org.apache.kafka.clients.producer.Callback but I did not see that happening
> either one or two are brokers not reachable.
>
> 2) Is jira ticket https://issues.apache.org/jira/browse/KAFKA-1788 will
> be merged to 0.8.2 ? This will give the ability to close the producer in
> event of lost connectivity to broker if io thread misbehave (does not end)
> ?
>
> Thanks for your help !
>
> Thanks,
> Bhavesh
>
Re: Follow-up On Important Issues for 0.8.2
Posted by Bhavesh Mistry <mi...@gmail.com>.
Adding User Community to see if any one knows behavior of Producer for
issue #1) and status of 2).
Thanks,
Bhavesh
On Fri, Jan 2, 2015 at 12:37 PM, Bhavesh Mistry <mi...@gmail.com>
wrote:
> Hi Kafka Dev Team,
>
> I am following-up with you guys regarding New (Java) Producer behavior in
> event of network or firewall rules. I just wanted to make Java Producer
> resilient of any network or firewall issues, and does not become
> single-point of failure in application:
>
> 1) Jira Issue https://issues.apache.org/jira/browse/KAFKA-1788
>
>
> https://issues.apache.org/jira/browse/KAFKA-1788?focusedCommentId=14259235&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14259235
>
> What should be the behavior of the Producer when it can not reach leader
> broker, but metadata reported broker is leader for that partition (via
> other broker) ? Should the record-error-rate be counted and Call Back
> should be called with error or not ?
>
> 1) *record-error-rate* metric remain zero despite following firewall
> rule. In my opinion, it should have called
> org.apache.kafka.clients.producer.Callback but I did not see that happening
> either one or two are brokers not reachable.
>
> 2) Is jira ticket https://issues.apache.org/jira/browse/KAFKA-1788 will
> be merged to 0.8.2 ? This will give the ability to close the producer in
> event of lost connectivity to broker if io thread misbehave (does not end)
> ?
>
> Thanks for your help !
>
> Thanks,
> Bhavesh
>