You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Tousif <to...@gmail.com> on 2015/05/01 04:14:51 UTC
Re: Netty connection issue
I get this error after few days of topology started running.
On 30 Apr 2015 22:18, "Supun Kamburugamuva" <su...@gmail.com> wrote:
> Are you getting this all the time?
>
> Thanks,
> Supun
> On Apr 30, 2015 5:52 AM, "Tousif" <to...@gmail.com> wrote:
>
>> I'm constantly getting this issue. Any help ?
>>
>> On Thu, Apr 23, 2015 at 12:15 PM, Tousif <to...@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> i see following error in one of the worker logs and message ack fails.
>>>
>>> b.s.m.n.Client [ERROR] discarding 1 messages because the Netty client
>>> to Netty-Client-server1/x.x.x.x:6703 is being closed
>>>
>>> how ever in Storm UI kafka-spout i see RuntimeException "No leader
>>> found for partition 2.
>>>
>>> all kafka brokers and zk quorum were up and running.
>>>
>>> Can anyone shed some light on what could be the reason ?
>>>
>>> --
>>>
>>>
>>> Regards
>>> Tousif Khazi
>>>
>>>
>>
>>
>> --
>>
>>
>> Regards
>> Tousif Khazi
>>
>>
Re: Netty connection issue
Posted by Tousif <to...@gmail.com>.
Thank you jeff,
Will check in kafka logs if everything is alright.
+1 for kafka-manager
On Mon, May 4, 2015 at 6:55 PM, Jeff Maass <JM...@cccis.com> wrote:
> that message is about kafka
> it is not saying that a kafka broker is down, but, a particular partition
> of a kafka topic is "down"
> two ways for you to see this:
> * /opt/kafka_2.9.2-0.8.1.1/bin/kafka-topics.sh --describe --zookeeper
> {a_node_from_kafkas_zookeeper_cluster}
> or
> * install and make use of yahoo's kafka manager:
> https://github.com/yahoo/kafka-manager
>
> you will need to reset the topic's partition to their preferred
> brokers. This command will reset every single topic. If you are only
> learning right now, go ahead and run this command, however, first, you
> probably want to figure out what happened in your kafka cluster in order to
> keep the problem from happening again.
>
> $KAFKA_HOME/bin/kafka-preferred-replica-election.sh --zookeeper
> {a_node_from_kafkas_zookeeper_cluster}
>
>
>
> From: Tousif <to...@gmail.com>
> Reply-To: "user@storm.apache.org" <us...@storm.apache.org>
> Date: 2015,Monday, May 4 at 02:27
> To: "user@storm.apache.org" <us...@storm.apache.org>
> Subject: Re: Netty connection issue
>
> Sorry for delayed response to your mail. I dont think IP address is
> changing as i use private IP's.
>
> On Fri, May 1, 2015 at 7:59 AM, Supun Kamburugamuva <su...@gmail.com>
> wrote:
>
>> Does your machines ips change after sometime?
>>
>> Thanks,
>> Supun..
>> On Apr 30, 2015 10:15 PM, "Tousif" <to...@gmail.com> wrote:
>>
>>> I get this error after few days of topology started running.
>>> On 30 Apr 2015 22:18, "Supun Kamburugamuva" <su...@gmail.com> wrote:
>>>
>>>> Are you getting this all the time?
>>>>
>>>> Thanks,
>>>> Supun
>>>> On Apr 30, 2015 5:52 AM, "Tousif" <to...@gmail.com> wrote:
>>>>
>>>>> I'm constantly getting this issue. Any help ?
>>>>>
>>>>> On Thu, Apr 23, 2015 at 12:15 PM, Tousif <to...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> i see following error in one of the worker logs and message ack
>>>>>> fails.
>>>>>>
>>>>>> b.s.m.n.Client [ERROR] discarding 1 messages because the Netty
>>>>>> client to Netty-Client-server1/x.x.x.x:6703 is being closed
>>>>>>
>>>>>> how ever in Storm UI kafka-spout i see RuntimeException "No leader
>>>>>> found for partition 2.
>>>>>>
>>>>>> all kafka brokers and zk quorum were up and running.
>>>>>>
>>>>>> Can anyone shed some light on what could be the reason ?
>>>>>>
>>>>>> --
>>>>>>
>>>>>>
>>>>>> Regards
>>>>>> Tousif Khazi
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>>
>>>>> Regards
>>>>> Tousif Khazi
>>>>>
>>>>>
>
>
> --
>
>
> Regards
> Tousif Khazi
>
>
--
Regards
Tousif Khazi
Re: Netty connection issue
Posted by Jeff Maass <JM...@cccis.com>.
that message is about kafka
it is not saying that a kafka broker is down, but, a particular partition of a kafka topic is "down"
two ways for you to see this:
* /opt/kafka_2.9.2-0.8.1.1/bin/kafka-topics.sh --describe --zookeeper {a_node_from_kafkas_zookeeper_cluster}
or
* install and make use of yahoo's kafka manager: https://github.com/yahoo/kafka-manager
you will need to reset the topic's partition to their preferred brokers. This command will reset every single topic. If you are only learning right now, go ahead and run this command, however, first, you probably want to figure out what happened in your kafka cluster in order to keep the problem from happening again.
$KAFKA_HOME/bin/kafka-preferred-replica-election.sh --zookeeper {a_node_from_kafkas_zookeeper_cluster}
From: Tousif <to...@gmail.com>>
Reply-To: "user@storm.apache.org<ma...@storm.apache.org>" <us...@storm.apache.org>>
Date: 2015,Monday, May 4 at 02:27
To: "user@storm.apache.org<ma...@storm.apache.org>" <us...@storm.apache.org>>
Subject: Re: Netty connection issue
Sorry for delayed response to your mail. I dont think IP address is changing as i use private IP's.
On Fri, May 1, 2015 at 7:59 AM, Supun Kamburugamuva <su...@gmail.com>> wrote:
Does your machines ips change after sometime?
Thanks,
Supun..
On Apr 30, 2015 10:15 PM, "Tousif" <to...@gmail.com>> wrote:
I get this error after few days of topology started running.
On 30 Apr 2015 22:18, "Supun Kamburugamuva" <su...@gmail.com>> wrote:
Are you getting this all the time?
Thanks,
Supun
On Apr 30, 2015 5:52 AM, "Tousif" <to...@gmail.com>> wrote:
I'm constantly getting this issue. Any help ?
On Thu, Apr 23, 2015 at 12:15 PM, Tousif <to...@gmail.com>> wrote:
Hello,
i see following error in one of the worker logs and message ack fails.
b.s.m.n.Client [ERROR] discarding 1 messages because the Netty client to Netty-Client-server1/x.x.x.x:6703 is being closed
how ever in Storm UI kafka-spout i see RuntimeException "No leader found for partition 2.
all kafka brokers and zk quorum were up and running.
Can anyone shed some light on what could be the reason ?
--
Regards
Tousif Khazi
--
Regards
Tousif Khazi
--
Regards
Tousif Khazi
Re: Netty connection issue
Posted by Tousif <to...@gmail.com>.
Sorry for delayed response to your mail. I dont think IP address is
changing as i use private IP's.
On Fri, May 1, 2015 at 7:59 AM, Supun Kamburugamuva <su...@gmail.com>
wrote:
> Does your machines ips change after sometime?
>
> Thanks,
> Supun..
> On Apr 30, 2015 10:15 PM, "Tousif" <to...@gmail.com> wrote:
>
>> I get this error after few days of topology started running.
>> On 30 Apr 2015 22:18, "Supun Kamburugamuva" <su...@gmail.com> wrote:
>>
>>> Are you getting this all the time?
>>>
>>> Thanks,
>>> Supun
>>> On Apr 30, 2015 5:52 AM, "Tousif" <to...@gmail.com> wrote:
>>>
>>>> I'm constantly getting this issue. Any help ?
>>>>
>>>> On Thu, Apr 23, 2015 at 12:15 PM, Tousif <to...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> i see following error in one of the worker logs and message ack fails.
>>>>>
>>>>> b.s.m.n.Client [ERROR] discarding 1 messages because the Netty client
>>>>> to Netty-Client-server1/x.x.x.x:6703 is being closed
>>>>>
>>>>> how ever in Storm UI kafka-spout i see RuntimeException "No leader
>>>>> found for partition 2.
>>>>>
>>>>> all kafka brokers and zk quorum were up and running.
>>>>>
>>>>> Can anyone shed some light on what could be the reason ?
>>>>>
>>>>> --
>>>>>
>>>>>
>>>>> Regards
>>>>> Tousif Khazi
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>>
>>>> Regards
>>>> Tousif Khazi
>>>>
>>>>
--
Regards
Tousif Khazi
Re: Netty connection issue
Posted by Supun Kamburugamuva <su...@gmail.com>.
Does your machines ips change after sometime?
Thanks,
Supun..
On Apr 30, 2015 10:15 PM, "Tousif" <to...@gmail.com> wrote:
> I get this error after few days of topology started running.
> On 30 Apr 2015 22:18, "Supun Kamburugamuva" <su...@gmail.com> wrote:
>
>> Are you getting this all the time?
>>
>> Thanks,
>> Supun
>> On Apr 30, 2015 5:52 AM, "Tousif" <to...@gmail.com> wrote:
>>
>>> I'm constantly getting this issue. Any help ?
>>>
>>> On Thu, Apr 23, 2015 at 12:15 PM, Tousif <to...@gmail.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> i see following error in one of the worker logs and message ack fails.
>>>>
>>>> b.s.m.n.Client [ERROR] discarding 1 messages because the Netty client
>>>> to Netty-Client-server1/x.x.x.x:6703 is being closed
>>>>
>>>> how ever in Storm UI kafka-spout i see RuntimeException "No leader
>>>> found for partition 2.
>>>>
>>>> all kafka brokers and zk quorum were up and running.
>>>>
>>>> Can anyone shed some light on what could be the reason ?
>>>>
>>>> --
>>>>
>>>>
>>>> Regards
>>>> Tousif Khazi
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>>
>>> Regards
>>> Tousif Khazi
>>>
>>>