You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by "Kaushal Kumar Singh (JIRA)" <ji...@apache.org> on 2016/05/26 16:34:13 UTC
[jira] [Created] (STORM-1867) Storm Topology Freezes
Kaushal Kumar Singh created STORM-1867:
------------------------------------------
Summary: Storm Topology Freezes
Key: STORM-1867
URL: https://issues.apache.org/jira/browse/STORM-1867
Project: Apache Storm
Issue Type: Bug
Components: storm-core
Affects Versions: 1.0.1
Environment: OS : CentOS Linux release 7.2.1511 (Core)
Storm 1.0.1
Kafka 0.9.0.1
zookeeper 3.4.8
Java 1.8.0_65
Reporter: Kaushal Kumar Singh
My storm topology freezes after few hours. I have a KafkaSpout, a parser bolt and an aggregator bolt.
After few hours I see KafkaSpout is not emitting anything. I checked the STORM UI, there were no errors. I also checked the kafka topic via kafka-console-consumer, I was able to consume but kafkaspout was emitting nothing.
I checked worker.log and there were no issue there.
Following is the some of the content of worker.log.
2016-05-26 03:45:16.252 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Refreshing partition manager connections
2016-05-26 03:45:16.260 o.a.s.k.DynamicBrokersReader [INFO] Read partition info from zookeeper: GlobalPartitionInformation{topic=l7v_flows, partitionMap={0=qagg2-storm:6667, 1=qagg3-storm:6667, 2=qagg1-storm:6667}}
2016-05-26 03:45:16.260 o.a.s.k.KafkaUtils [INFO] Task [3/3] assigned [Partition{host=qagg1-storm:6667, topic=l7v_flows, partition=2}]
2016-05-26 03:45:16.260 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Deleted partition managers: []
2016-05-26 03:45:16.260 o.a.s.k.ZkCoordinator [INFO] Task [3/3] New partition managers: []
2016-05-26 03:45:16.260 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Finished refreshing
2016-05-26 03:47:16.252 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Refreshing partition manager connections
2016-05-26 03:47:16.260 o.a.s.k.DynamicBrokersReader [INFO] Read partition info from zookeeper: GlobalPartitionInformation{topic=l7v_flows, partitionMap={0=qagg2-storm:6667, 1=qagg3-storm:6667, 2=qagg1-storm:6667}}
2016-05-26 03:47:16.260 o.a.s.k.KafkaUtils [INFO] Task [3/3] assigned [Partition{host=qagg1-storm:6667, topic=l7v_flows, partition=2}]
2016-05-26 03:47:16.261 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Deleted partition managers: []
2016-05-26 03:47:16.261 o.a.s.k.ZkCoordinator [INFO] Task [3/3] New partition managers: []
2016-05-26 03:47:16.261 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Finished refreshing
2016-05-26 03:49:16.252 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Refreshing partition manager connections
2016-05-26 03:49:16.259 o.a.s.k.DynamicBrokersReader [INFO] Read partition info from zookeeper: GlobalPartitionInformation{topic=l7v_flows, partitionMap={0=qagg2-storm:6667, 1=qagg3-storm:6667, 2=qagg1-storm:6667}}
2016-05-26 03:49:16.260 o.a.s.k.KafkaUtils [INFO] Task [3/3] assigned [Partition{host=qagg1-storm:6667, topic=l7v_flows, partition=2}]
2016-05-26 03:49:16.260 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Deleted partition managers: []
2016-05-26 03:49:16.260 o.a.s.k.ZkCoordinator [INFO] Task [3/3] New partition managers: []
2016-05-26 03:49:16.260 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Finished refreshing
2016-05-26 03:51:16.254 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Refreshing partition manager connections
2016-05-26 03:51:16.259 o.a.s.k.DynamicBrokersReader [INFO] Read partition info from zookeeper: GlobalPartitionInformation{topic=l7v_flows, partitionMap={0=qagg2-storm:6667, 1=qagg3-storm:6667, 2=qagg1-storm:6667}}
2016-05-26 03:51:16.260 o.a.s.k.KafkaUtils [INFO] Task [3/3] assigned [Partition{host=qagg1-storm:6667, topic=l7v_flows, partition=2}]
2016-05-26 03:51:16.260 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Deleted partition managers: []
2016-05-26 03:51:16.260 o.a.s.k.ZkCoordinator [INFO] Task [3/3] New partition managers: []
2016-05-26 03:51:16.260 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Finished refreshing
2016-05-26 03:53:16.254 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Refreshing partition manager connections
2016-05-26 03:53:16.260 o.a.s.k.DynamicBrokersReader [INFO] Read partition info from zookeeper: GlobalPartitionInformation{topic=l7v_flows, partitionMap={0=qagg2-storm:6667, 1=qagg3-storm:6667, 2=qagg1-storm:6667}}
2016-05-26 03:53:16.260 o.a.s.k.KafkaUtils [INFO] Task [3/3] assigned [Partition{host=qagg1-storm:6667, topic=l7v_flows, partition=2}]
2016-05-26 03:53:16.260 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Deleted partition managers: []
2016-05-26 03:53:16.260 o.a.s.k.ZkCoordinator [INFO] Task [3/3] New partition managers: []
2016-05-26 03:53:16.260 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Finished refreshing
2016-05-26 03:55:16.255 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Refreshing partition manager connections
2016-05-26 03:55:16.265 o.a.s.k.DynamicBrokersReader [INFO] Read partition info from zookeeper: GlobalPartitionInformation{topic=l7v_flows, partitionMap={0=qagg2-storm:6667, 1=qagg3-storm:6667, 2=qagg1-storm:6667}}
2016-05-26 03:55:16.265 o.a.s.k.KafkaUtils [INFO] Task [3/3] assigned [Partition{host=qagg1-storm:6667, topic=l7v_flows, partition=2}]
2016-05-26 03:55:16.266 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Deleted partition managers: []
2016-05-26 03:55:16.266 o.a.s.k.ZkCoordinator [INFO] Task [3/3] New partition managers: []
2016-05-26 03:55:16.266 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Finished refreshing
2016-05-26 03:57:16.255 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Refreshing partition manager connections
2016-05-26 03:57:16.261 o.a.s.k.DynamicBrokersReader [INFO] Read partition info from zookeeper: GlobalPartitionInformation{topic=l7v_flows, partitionMap={0=qagg2-storm:6667, 1=qagg3-storm:6667, 2=qagg1-storm:6667}}
2016-05-26 03:57:16.261 o.a.s.k.KafkaUtils [INFO] Task [3/3] assigned [Partition{host=qagg1-storm:6667, topic=l7v_flows, partition=2}]
2016-05-26 03:57:16.262 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Deleted partition managers: []
2016-05-26 03:57:16.262 o.a.s.k.ZkCoordinator [INFO] Task [3/3] New partition managers: []
2016-05-26 03:57:16.262 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Finished refreshing
2016-05-26 03:59:16.255 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Refreshing partition manager connections
2016-05-26 03:59:16.260 o.a.s.k.DynamicBrokersReader [INFO] Read partition info from zookeeper: GlobalPartitionInformation{topic=l7v_flows, partitionMap={0=qagg2-storm:6667, 1=qagg3-storm:6667, 2=qagg1-storm:6667}}
2016-05-26 03:59:16.261 o.a.s.k.KafkaUtils [INFO] Task [3/3] assigned [Partition{host=qagg1-storm:6667, topic=l7v_flows, partition=2}]
2016-05-26 03:59:16.261 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Deleted partition managers: []
2016-05-26 03:59:16.261 o.a.s.k.ZkCoordinator [INFO] Task [3/3] New partition managers: []
2016-05-26 03:59:16.261 o.a.s.k.ZkCoordinator [INFO] Task [3/3] Finished refreshing
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)