Fwd: Fwd: suddenly kafka cluster shutdown

[Jose Raul Perez Rodriguez <jo...@gmail.com>]

Hi kafka user and devs,

I have a kafka cluster on AWS EMR instances using de zookeeper that
cames with EMR,

The problem is that after some days running and working good, the
cluster shutdown, every node shutdown showing logs messages like the one
below.

That is happening in the las two month 3 times, with apparently no
reasons because no error is shown, always with version kafka 11  .


[2017-11-22 16:00:00,002] INFO [GroupCoordinator 1003]: Member
consumer-2-9e7145cf-9f7f-4056-b9aa-8cab564b6867 in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:00:00,003] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 7
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:00:00,003] INFO [GroupCoordinator 1003]: Group
telecomming-stream-0 with generation 8 is now empty
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:02:10,957] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 8
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:02:10,958] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 9 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:02:11,037] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 9
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:02:32,409] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 9
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:02:32,410] INFO [GroupCoordinator 1003]: Group
telecomming-stream-0 with generation 10 is now empty
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:04:53,252] INFO [Group Metadata Manager on Broker 1003]:
Group telecomming-stream-0 transitioned to Dead in generation 10
(kafka.coordinator.group.GroupMetadataManager)
[2017-11-22 16:04:53,252] INFO [Group Metadata Manager on Broker 1003]:
Removed 0 expired offsets in 0 milliseconds.
(kafka.coordinator.group.GroupMetadataManager)
[2017-11-22 16:12:47,441] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 0
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:12:47,442] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 1 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:12:47,442] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 1
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:12:47,446] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 2 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:12:47,529] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 2
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:14:30,009] INFO [GroupCoordinator 1003]: Member
consumer-1-9d40ffcc-0f75-44f2-b9db-b34dfd47fd6c in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:14:30,009] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 2
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:14:53,252] INFO [Group Metadata Manager on Broker 1003]:
Removed 0 expired offsets in 0 milliseconds.
(kafka.coordinator.group.GroupMetadataManager)
[2017-11-22 16:15:00,010] INFO [GroupCoordinator 1003]: Group
telecomming-stream-0 with generation 3 is now empty
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:15:30,002] INFO [GroupCoordinator 1003]: Member
consumer-2-653f3962-8259-47e9-a8dc-4b23d85f2c80 in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:15:30,003] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 3
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:15:30,003] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 4 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:15:30,004] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 4
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:16:00,005] INFO [GroupCoordinator 1003]: Member
consumer-2-5c6d78ef-1422-4123-b0fe-76de4745bb6e in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:16:00,005] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 4
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:16:00,005] INFO [GroupCoordinator 1003]: Group
telecomming-stream-0 with generation 5 is now empty
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:16:30,002] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 5
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:16:30,002] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 6 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:16:30,003] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 6
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:17:00,509] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 6
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:17:30,003] INFO [GroupCoordinator 1003]: Member
consumer-2-f8be8578-43b2-447a-8a75-92e9278d395b in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:17:30,004] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 7 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:17:30,005] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 7
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:18:00,007] INFO [GroupCoordinator 1003]: Member
consumer-1-b27cb2d8-083b-4646-8da0-37658236b1cc in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:18:00,007] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 7
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:18:00,007] INFO [GroupCoordinator 1003]: Group
telecomming-stream-0 with generation 8 is now empty
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:18:30,508] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 8
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:18:30,509] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 9 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:18:30,510] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 9
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:19:00,002] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 9
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:19:00,512] INFO [GroupCoordinator 1003]: Member
consumer-1-8c6546a2-27ab-487a-8099-f32bc2942682 in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:19:00,512] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 10 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:19:00,513] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 10
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:20:00,002] INFO [GroupCoordinator 1003]: Member
consumer-2-6e281ac3-60a9-4e50-b285-6334d60ca07f in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:20:00,002] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 10
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:20:00,002] INFO [GroupCoordinator 1003]: Group
telecomming-stream-0 with generation 11 is now empty
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:20:30,002] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 11
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:20:30,002] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 12 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:20:30,003] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 12
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:20:30,511] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 12
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:21:00,005] INFO [GroupCoordinator 1003]: Member
consumer-2-c3cbd77f-5eae-4151-8748-5b85e71d4634 in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:21:00,005] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 13 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:21:30,005] INFO [GroupCoordinator 1003]: Member
consumer-1-50a70da3-1d82-4e57-8d35-4934da36ef04 in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:21:30,005] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 13
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:21:30,005] INFO [GroupCoordinator 1003]: Group
telecomming-stream-0 with generation 14 is now empty
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:23:51,941] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 14
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:23:51,942] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 15 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:23:52,017] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 15
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:24:53,252] INFO [Group Metadata Manager on Broker 1003]:
Removed 0 expired offsets in 0 milliseconds.
(kafka.coordinator.group.GroupMetadataManager)
[2017-11-22 16:25:30,008] INFO [GroupCoordinator 1003]: Member
consumer-2-30be21c3-d41c-4700-b242-358182a67361 in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:25:30,008] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 15
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:26:00,009] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 16 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:26:00,010] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 16
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:26:30,003] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 16
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:26:30,003] INFO [GroupCoordinator 1003]: Member
consumer-1-e9b236fa-392f-4209-a053-e018363d14f3 in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:26:30,011] INFO [GroupCoordinator 1003]: Member
consumer-2-b895525d-b2e3-4871-88fd-6b78ba42ebef in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:26:30,011] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 17 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:26:30,012] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 17
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:28:00,508] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 17
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:28:30,003] INFO [GroupCoordinator 1003]: Member
consumer-1-66200dd6-9da9-4c5d-bd81-2585e62a7b15 in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:28:30,004] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 18 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:28:30,005] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 18
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:29:00,003] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 18
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:29:00,007] INFO [GroupCoordinator 1003]: Member
consumer-2-df47d6c8-d144-4687-aaf7-15874b603058 in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:29:00,007] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 19 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:29:00,008] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 19
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:30:30,509] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 19
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:31:00,004] INFO [GroupCoordinator 1003]: Member
consumer-1-aa52e8cb-2d2b-4dae-96fe-49e6dbd5571b in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:31:00,004] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 20 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:31:00,006] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 20
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:31:30,006] INFO [GroupCoordinator 1003]: Member
consumer-2-ac6f1e9f-0d24-4fc4-ba74-f74a9a196f14 in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:31:30,006] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 20
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:31:30,006] INFO [GroupCoordinator 1003]: Group
telecomming-stream-0 with generation 21 is now empty
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:32:30,002] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 21
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:32:30,003] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 22 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:32:30,004] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 22
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:34:00,506] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 22
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:34:30,002] INFO [GroupCoordinator 1003]: Member
consumer-1-4de600bb-9fbd-4cd5-ba61-9f9e8b23d625 in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:34:30,002] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 23 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:34:30,003] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 23
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:34:53,252] INFO [Group Metadata Manager on Broker 1003]:
Removed 0 expired offsets in 0 milliseconds.
(kafka.coordinator.group.GroupMetadataManager)
[2017-11-22 16:35:00,004] INFO [GroupCoordinator 1003]: Member
consumer-2-7f9b0769-83be-4c13-bb96-7d44892774df in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:35:00,004] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 23
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:35:00,005] INFO [GroupCoordinator 1003]: Group
telecomming-stream-0 with generation 24 is now empty
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:36:00,003] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 24
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:36:00,003] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 25 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:36:00,004] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 25
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:37:30,002] INFO [GroupCoordinator 1003]: Member
consumer-1-8d389137-fc4f-4b3f-9a53-2ef7fed69019 in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:37:30,002] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 25
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:37:30,002] INFO [GroupCoordinator 1003]: Group
telecomming-stream-0 with generation 26 is now empty
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:37:30,506] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 26
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:37:30,507] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 27 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:37:30,507] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 27
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:38:00,002] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 27
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:38:00,508] INFO [GroupCoordinator 1003]: Member
consumer-2-cbbf88a2-9f69-425c-a9ff-249b4d8e8a76 in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:38:00,508] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 28 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:38:00,509] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 28
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:39:30,507] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 28
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:40:00,003] INFO [GroupCoordinator 1003]: Member
consumer-1-9bd3ef54-9da3-4219-b65d-b1cf85e8ae9c in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:40:00,004] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 29 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:40:00,005] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 29
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:40:30,007] INFO [GroupCoordinator 1003]: Member
consumer-2-950ac352-851c-4139-b85a-79844312bcaa in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:40:30,007] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 29
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:40:30,007] INFO [GroupCoordinator 1003]: Group
telecomming-stream-0 with generation 30 is now empty
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:41:30,002] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 30
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:41:30,003] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 31 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:41:30,003] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 31
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:43:00,506] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 31
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:43:30,004] INFO [GroupCoordinator 1003]: Member
consumer-1-dc2b2ae4-462a-4156-9cc0-51b34aab174f in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:43:30,004] INFO [GroupCoordinator 1003]: Stabilized group
telecomming-stream-0 generation 32 (__consumer_offsets-28)
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:43:30,005] INFO [GroupCoordinator 1003]: Assignment
received from leader for group telecomming-stream-0 for generation 32
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:44:00,008] INFO [GroupCoordinator 1003]: Member
consumer-2-10460c09-eb13-425b-b5f8-f49c6483bf9c in group
telecomming-stream-0 has failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:44:00,008] INFO [GroupCoordinator 1003]: Preparing to
rebalance group telecomming-stream-0 with old generation 32
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:44:00,009] INFO [GroupCoordinator 1003]: Group
telecomming-stream-0 with generation 33 is now empty
(__consumer_offsets-28) (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:44:41,434] INFO [Kafka Server 1003], shutting down
(kafka.server.KafkaServer)
[2017-11-22 16:44:41,436] INFO [Kafka Server 1003], Starting controlled
shutdown (kafka.server.KafkaServer)
[2017-11-22 16:44:41,471] INFO [Kafka Server 1003], Controlled shutdown
succeeded (kafka.server.KafkaServer)
[2017-11-22 16:44:41,473] INFO [Socket Server on Broker 1003], Shutting
down (kafka.network.SocketServer)
[2017-11-22 16:44:41,481] INFO [Socket Server on Broker 1003], Shutdown
completed (kafka.network.SocketServer)
[2017-11-22 16:44:41,482] INFO [Kafka Request Handler on Broker 1003],
shutting down (kafka.server.KafkaRequestHandlerPool)
[2017-11-22 16:44:41,483] INFO [Kafka Request Handler on Broker 1003],
shut down completely (kafka.server.KafkaRequestHandlerPool)
[2017-11-22 16:44:41,486] INFO [ThrottledRequestReaper-Fetch]: Shutting
down (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-11-22 16:44:42,039] INFO [ThrottledRequestReaper-Fetch]: Shutdown
completed (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-11-22 16:44:42,039] INFO [ThrottledRequestReaper-Fetch]: Stopped
(kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-11-22 16:44:42,040] INFO [ThrottledRequestReaper-Produce]:
Shutting down (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-11-22 16:44:43,039] INFO [ThrottledRequestReaper-Produce]: Stopped
(kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-11-22 16:44:43,039] INFO [ThrottledRequestReaper-Produce]:
Shutdown completed (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-11-22 16:44:43,039] INFO [ThrottledRequestReaper-Request]:
Shutting down (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-11-22 16:44:44,039] INFO [ThrottledRequestReaper-Request]: Stopped
(kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-11-22 16:44:44,039] INFO [ThrottledRequestReaper-Request]:
Shutdown completed (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-11-22 16:44:44,040] INFO [KafkaApi-1003] Shutdown complete.
(kafka.server.KafkaApis)
[2017-11-22 16:44:44,040] INFO [ExpirationReaper-1003-topic]: Shutting
down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,097] INFO [ExpirationReaper-1003-topic]: Stopped
(kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,097] INFO [ExpirationReaper-1003-topic]: Shutdown
completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,098] INFO [Transaction Coordinator 1003]: Shutting
down. (kafka.coordinator.transaction.TransactionCoordinator)
[2017-11-22 16:44:44,099] INFO [ProducerId Manager 1003]: Shutdown
complete: last producerId assigned 14000
(kafka.coordinator.transaction.ProducerIdManager)
[2017-11-22 16:44:44,099] INFO [Transaction State Manager 1003]:
Shutdown complete (kafka.coordinator.transaction.TransactionStateManager)
[2017-11-22 16:44:44,099] INFO [Transaction Marker Channel Manager
1003]: Shutting down
(kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2017-11-22 16:44:44,099] INFO [Transaction Marker Channel Manager
1003]: Stopped
(kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2017-11-22 16:44:44,099] INFO [Transaction Marker Channel Manager
1003]: Shutdown completed
(kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2017-11-22 16:44:44,100] INFO [Transaction Coordinator 1003]: Shutdown
complete. (kafka.coordinator.transaction.TransactionCoordinator)
[2017-11-22 16:44:44,100] INFO [GroupCoordinator 1003]: Shutting down.
(kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:44:44,100] INFO [ExpirationReaper-1003-Heartbeat]:
Shutting down
(kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,235] INFO [ExpirationReaper-1003-Heartbeat]:
Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,235] INFO [ExpirationReaper-1003-Heartbeat]:
Shutdown completed
(kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,235] INFO [ExpirationReaper-1003-Rebalance]:
Shutting down
(kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,426] INFO [ExpirationReaper-1003-Rebalance]:
Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,426] INFO [ExpirationReaper-1003-Rebalance]:
Shutdown completed
(kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,426] INFO [GroupCoordinator 1003]: Shutdown
complete. (kafka.coordinator.group.GroupCoordinator)
[2017-11-22 16:44:44,427] INFO [Replica Manager on Broker 1003]:
Shutting down (kafka.server.ReplicaManager)
[2017-11-22 16:44:44,427] INFO [ReplicaFetcherManager on broker 1003]
shutting down (kafka.server.ReplicaFetcherManager)
[2017-11-22 16:44:44,428] INFO [ReplicaFetcherManager on broker 1003]
shutdown completed (kafka.server.ReplicaFetcherManager)
[2017-11-22 16:44:44,428] INFO [ExpirationReaper-1003-Fetch]: Shutting
down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,602] INFO [ExpirationReaper-1003-Fetch]: Stopped
(kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,602] INFO [ExpirationReaper-1003-Fetch]: Shutdown
completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,602] INFO [ExpirationReaper-1003-Produce]: Shutting
down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,697] INFO [ExpirationReaper-1003-Produce]: Stopped
(kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,697] INFO [ExpirationReaper-1003-Produce]: Shutdown
completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,697] INFO [ExpirationReaper-1003-DeleteRecords]:
Shutting down
(kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,897] INFO [ExpirationReaper-1003-DeleteRecords]:
Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,897] INFO [ExpirationReaper-1003-DeleteRecords]:
Shutdown completed
(kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-11-22 16:44:44,899] INFO [Replica Manager on Broker 1003]: Shut
down completely (kafka.server.ReplicaManager)
[2017-11-22 16:44:44,900] INFO Shutting down. (kafka.log.LogManager)
[2017-11-22 16:44:44,920] INFO Shutdown complete. (kafka.log.LogManager)
[2017-11-22 16:44:44,924] INFO Terminate ZkClient event thread.
(org.I0Itec.zkclient.ZkEventThread)
[2017-11-22 16:44:44,925] INFO Session: 0x15fbb0e6caf0045 closed
(org.apache.zookeeper.ZooKeeper)
[2017-11-22 16:44:44,927] INFO EventThread shut down for session:
0x15fbb0e6caf0045 (org.apache.zookeeper.ClientCnxn)
[2017-11-22 16:44:44,929] INFO [Kafka Server 1003], shut down completed
(kafka.server.KafkaServer)


Thanks, a lot

Jr