You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "John Fung (JIRA)" <ji...@apache.org> on 2012/11/02 18:05:11 UTC
[jira] [Created] (KAFKA-599) SimpleConsumerShell ONLY connects to
the first host in the broker-list string to fetch topic metadata
John Fung created KAFKA-599:
-------------------------------
Summary: SimpleConsumerShell ONLY connects to the first host in the broker-list string to fetch topic metadata
Key: KAFKA-599
URL: https://issues.apache.org/jira/browse/KAFKA-599
Project: Kafka
Issue Type: Bug
Reporter: John Fung
Priority: Critical
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (KAFKA-599) SimpleConsumerShell ONLY
connects to the first host in the broker-list string to fetch topic
metadata
Posted by "John Fung (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13489541#comment-13489541 ]
John Fung edited comment on KAFKA-599 at 11/2/12 5:06 PM:
----------------------------------------------------------
** This is showing in testcase_0253 in which the first broker (out of the 3-brokers cluster) is down. SimpleConsumerShell doesn't seem to try to connect to other available brokers to fetch topic metadata.
1. System Test output showing the command line arguments of SimpleConsumerShell
2012-11-02 06:52:15,728 - INFO - starting debug consumer for replica on [host_0997:9091] partition [0] (kafka_system_test_utils)
2012-11-02 06:52:15,729 - DEBUG - executing command: [ssh host_0996.prod 'JAVA_HOME=/export/apps/jdk/JDK-1_6_0_21 /kafka_08_replication_system_test/bin/kafka-run-class.sh kafka.tools.SimpleConsumerShell --broker-list host_0997:9091,host_0998:9092,host_0999:9093 --topic test_1 --partition 0 --replica 1 --offset -2 --no-wait-at-logend
2. SimpleConsumerShell keeps trying on the first host to fetch topic metadata:
[2012-11-02 06:52:16,290] INFO Getting topic metatdata... (kafka.tools.SimpleConsumerShell$)
[2012-11-02 06:52:16,352] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:16,361] ERROR Producer connection to host_0997:9091 unsuccessful (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:16,364] WARN fetching topic metadata for topics [Set(test_1)] from broker [id:0,creatorId:host_0997-1351839136304,host:host_0997,port:9091] failed (kafka.client.ClientUtils$)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:16,364] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:16,366] INFO Connected to host_0998:9092 for producing (kafka.producer.SyncProducer)
[2012-11-02 06:52:16,412] INFO Disconnecting from host_0998:9092 (kafka.producer.SyncProducer)
[2012-11-02 06:52:18,324] INFO Getting topic metatdata... (kafka.tools.SimpleConsumerShell$)
[2012-11-02 06:52:18,386] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:18,395] ERROR Producer connection to host_0997:9091 unsuccessful (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:18,398] WARN fetching topic metadata for topics [Set(test_1)] from broker [id:0,creatorId:host_0997-1351839138338,host:host_0997,port:9091] failed (kafka.client.ClientUtils$)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:18,399] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:18,401] INFO Connected to host_0998:9092 for producing (kafka.producer.SyncProducer)
[2012-11-02 06:52:18,446] INFO Disconnecting from host_0998:9092 (kafka.producer.SyncProducer)
[2012-11-02 06:52:20,355] INFO Getting topic metatdata... (kafka.tools.SimpleConsumerShell$)
[2012-11-02 06:52:20,417] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:20,426] ERROR Producer connection to host_0997:9091 unsuccessful (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
was (Author: jfung):
** This is showing in testcase_0253 in which the first broker (out of the 3-brokers cluster) is down. SimpleConsumerShell doesn't seem to try to connect to other available brokers to fetch topic metadata.
1. System Test output showing the command line arguments of SimpleConsumerShell
2012-11-02 06:52:15,728 - INFO - starting debug consumer for replica on [host_0997:9091] partition [0] (kafka_system_test_utils)
2012-11-02 06:52:15,729 - DEBUG - executing command: [ssh host_0996.prod 'JAVA_HOME=/export/apps/jdk/JDK-1_6_0_21 /kafka_08_replication_system_test/bin/kafka-run-class.sh kafka.tools.SimpleConsumerShell --broker-list host_0997:9091,host_0998:9092,host_0999:9093 --topic test_1 --partition 0 --replica 1 --offset -2 --no-wait-at-logend >> /mnt/u001/kafka_08_replication_system_test/system_test/replication_testsuite/testcase_0253/logs/console_consumer-5/simple_consumer_1.log & echo pid:$! > /mnt/u001/kafka_08_replication_system_test/system_test/replication_testsuite/testcase_0253/logs/console_consumer-5/entity_5_pid'] (kafka_system_test_utils)
2. SimpleConsumerShell keeps trying on the first host to fetch topic metadata:
[2012-11-02 06:52:16,290] INFO Getting topic metatdata... (kafka.tools.SimpleConsumerShell$)
[2012-11-02 06:52:16,352] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:16,361] ERROR Producer connection to host_0997:9091 unsuccessful (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:16,364] WARN fetching topic metadata for topics [Set(test_1)] from broker [id:0,creatorId:host_0997-1351839136304,host:host_0997,port:9091] failed (kafka.client.ClientUtils$)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:16,364] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:16,366] INFO Connected to host_0998:9092 for producing (kafka.producer.SyncProducer)
[2012-11-02 06:52:16,412] INFO Disconnecting from host_0998:9092 (kafka.producer.SyncProducer)
[2012-11-02 06:52:18,324] INFO Getting topic metatdata... (kafka.tools.SimpleConsumerShell$)
[2012-11-02 06:52:18,386] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:18,395] ERROR Producer connection to host_0997:9091 unsuccessful (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:18,398] WARN fetching topic metadata for topics [Set(test_1)] from broker [id:0,creatorId:host_0997-1351839138338,host:host_0997,port:9091] failed (kafka.client.ClientUtils$)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:18,399] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:18,401] INFO Connected to host_0998:9092 for producing (kafka.producer.SyncProducer)
[2012-11-02 06:52:18,446] INFO Disconnecting from host_0998:9092 (kafka.producer.SyncProducer)
[2012-11-02 06:52:20,355] INFO Getting topic metatdata... (kafka.tools.SimpleConsumerShell$)
[2012-11-02 06:52:20,417] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:20,426] ERROR Producer connection to host_0997:9091 unsuccessful (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
> SimpleConsumerShell ONLY connects to the first host in the broker-list string to fetch topic metadata
> -----------------------------------------------------------------------------------------------------
>
> Key: KAFKA-599
> URL: https://issues.apache.org/jira/browse/KAFKA-599
> Project: Kafka
> Issue Type: Bug
> Reporter: John Fung
> Priority: Critical
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (KAFKA-599) SimpleConsumerShell ONLY
connects to the first host in the broker-list string to fetch topic
metadata
Posted by "John Fung (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13489541#comment-13489541 ]
John Fung edited comment on KAFKA-599 at 11/2/12 5:05 PM:
----------------------------------------------------------
** This is showing in testcase_0253 in which the first broker (out of the 3-brokers cluster) is down. SimpleConsumerShell doesn't seem to try to connect to other available brokers to fetch topic metadata.
1. System Test output showing the command line arguments of SimpleConsumerShell
2012-11-02 06:52:15,728 - INFO - starting debug consumer for replica on [host_0997:9091] partition [0] (kafka_system_test_utils)
2012-11-02 06:52:15,729 - DEBUG - executing command: [ssh host_0996.prod 'JAVA_HOME=/export/apps/jdk/JDK-1_6_0_21 /kafka_08_replication_system_test/bin/kafka-run-class.sh kafka.tools.SimpleConsumerShell --broker-list host_0997:9091,host_0998:9092,host_0999:9093 --topic test_1 --partition 0 --replica 1 --offset -2 --no-wait-at-logend >> /mnt/u001/kafka_08_replication_system_test/system_test/replication_testsuite/testcase_0253/logs/console_consumer-5/simple_consumer_1.log & echo pid:$! > /mnt/u001/kafka_08_replication_system_test/system_test/replication_testsuite/testcase_0253/logs/console_consumer-5/entity_5_pid'] (kafka_system_test_utils)
2. SimpleConsumerShell keeps trying on the first host to fetch topic metadata:
[2012-11-02 06:52:16,290] INFO Getting topic metatdata... (kafka.tools.SimpleConsumerShell$)
[2012-11-02 06:52:16,352] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:16,361] ERROR Producer connection to host_0997:9091 unsuccessful (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:16,364] WARN fetching topic metadata for topics [Set(test_1)] from broker [id:0,creatorId:host_0997-1351839136304,host:host_0997,port:9091] failed (kafka.client.ClientUtils$)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:16,364] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:16,366] INFO Connected to host_0998:9092 for producing (kafka.producer.SyncProducer)
[2012-11-02 06:52:16,412] INFO Disconnecting from host_0998:9092 (kafka.producer.SyncProducer)
[2012-11-02 06:52:18,324] INFO Getting topic metatdata... (kafka.tools.SimpleConsumerShell$)
[2012-11-02 06:52:18,386] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:18,395] ERROR Producer connection to host_0997:9091 unsuccessful (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:18,398] WARN fetching topic metadata for topics [Set(test_1)] from broker [id:0,creatorId:host_0997-1351839138338,host:host_0997,port:9091] failed (kafka.client.ClientUtils$)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:18,399] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:18,401] INFO Connected to host_0998:9092 for producing (kafka.producer.SyncProducer)
[2012-11-02 06:52:18,446] INFO Disconnecting from host_0998:9092 (kafka.producer.SyncProducer)
[2012-11-02 06:52:20,355] INFO Getting topic metatdata... (kafka.tools.SimpleConsumerShell$)
[2012-11-02 06:52:20,417] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:20,426] ERROR Producer connection to host_0997:9091 unsuccessful (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
was (Author: jfung):
** This is showing in testcase_0253 in which the first broker (out of the 3-brokers cluster) is down. SimpleConsumerShell doesn't seem to try to connect to other available brokers to fetch topic metadata.
1. System Test output showing the command line arguments of SimpleConsumerShell
2012-11-02 06:52:15,728 - INFO - starting debug consumer for replica on [host_0997:9091] partition [0] (kafka_system_test_utils)
2012-11-02 06:52:15,729 - DEBUG - executing command: [ssh host_0996.prod 'JAVA_HOME=/export/apps/jdk/JDK-1_6_0_21 /mnt/u001/kafka_08_replication_system_test/bin/kafka-run-class.sh kafka.tools.SimpleConsumerShell --broker-list host_0997:9091,host_0998:9092,host_0999:9093 --topic test_1 --partition 0 --replica 1 --offset -2 --no-wait-at-logend >> /mnt/u001/kafka_08_replication_system_test/system_test/replication_testsuite/testcase_0253/logs/console_consumer-5/simple_consumer_1.log & echo pid:$! > /mnt/u001/kafka_08_replication_system_test/system_test/replication_testsuite/testcase_0253/logs/console_consumer-5/entity_5_pid'] (kafka_system_test_utils)
2. SimpleConsumerShell keeps trying on the first host to fetch topic metadata:
[2012-11-02 06:52:16,290] INFO Getting topic metatdata... (kafka.tools.SimpleConsumerShell$)
[2012-11-02 06:52:16,352] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:16,361] ERROR Producer connection to host_0997:9091 unsuccessful (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:16,364] WARN fetching topic metadata for topics [Set(test_1)] from broker [id:0,creatorId:host_0997-1351839136304,host:host_0997,port:9091] failed (kafka.client.ClientUtils$)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:16,364] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:16,366] INFO Connected to host_0998:9092 for producing (kafka.producer.SyncProducer)
[2012-11-02 06:52:16,412] INFO Disconnecting from host_0998:9092 (kafka.producer.SyncProducer)
[2012-11-02 06:52:18,324] INFO Getting topic metatdata... (kafka.tools.SimpleConsumerShell$)
[2012-11-02 06:52:18,386] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:18,395] ERROR Producer connection to host_0997:9091 unsuccessful (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:18,398] WARN fetching topic metadata for topics [Set(test_1)] from broker [id:0,creatorId:host_0997-1351839138338,host:host_0997,port:9091] failed (kafka.client.ClientUtils$)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:18,399] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:18,401] INFO Connected to host_0998:9092 for producing (kafka.producer.SyncProducer)
[2012-11-02 06:52:18,446] INFO Disconnecting from host_0998:9092 (kafka.producer.SyncProducer)
[2012-11-02 06:52:20,355] INFO Getting topic metatdata... (kafka.tools.SimpleConsumerShell$)
[2012-11-02 06:52:20,417] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:20,426] ERROR Producer connection to host_0997:9091 unsuccessful (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
> SimpleConsumerShell ONLY connects to the first host in the broker-list string to fetch topic metadata
> -----------------------------------------------------------------------------------------------------
>
> Key: KAFKA-599
> URL: https://issues.apache.org/jira/browse/KAFKA-599
> Project: Kafka
> Issue Type: Bug
> Reporter: John Fung
> Priority: Critical
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-599) SimpleConsumerShell ONLY connects to
the first host in the broker-list string to fetch topic metadata
Posted by "John Fung (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13489541#comment-13489541 ]
John Fung commented on KAFKA-599:
---------------------------------
** This is showing in testcase_0253 in which the first broker (out of the 3-brokers cluster) is down. SimpleConsumerShell doesn't seem to try to connect to other available brokers to fetch topic metadata.
1. System Test output showing the command line arguments of SimpleConsumerShell
2012-11-02 06:52:15,728 - INFO - starting debug consumer for replica on [host_0997:9091] partition [0] (kafka_system_test_utils)
2012-11-02 06:52:15,729 - DEBUG - executing command: [ssh host_0996.prod 'JAVA_HOME=/export/apps/jdk/JDK-1_6_0_21 /mnt/u001/kafka_08_replication_system_test/bin/kafka-run-class.sh kafka.tools.SimpleConsumerShell --broker-list host_0997:9091,host_0998:9092,host_0999:9093 --topic test_1 --partition 0 --replica 1 --offset -2 --no-wait-at-logend >> /mnt/u001/kafka_08_replication_system_test/system_test/replication_testsuite/testcase_0253/logs/console_consumer-5/simple_consumer_1.log & echo pid:$! > /mnt/u001/kafka_08_replication_system_test/system_test/replication_testsuite/testcase_0253/logs/console_consumer-5/entity_5_pid'] (kafka_system_test_utils)
2. SimpleConsumerShell keeps trying on the first host to fetch topic metadata:
[2012-11-02 06:52:16,290] INFO Getting topic metatdata... (kafka.tools.SimpleConsumerShell$)
[2012-11-02 06:52:16,352] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:16,361] ERROR Producer connection to host_0997:9091 unsuccessful (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:16,364] WARN fetching topic metadata for topics [Set(test_1)] from broker [id:0,creatorId:host_0997-1351839136304,host:host_0997,port:9091] failed (kafka.client.ClientUtils$)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:16,364] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:16,366] INFO Connected to host_0998:9092 for producing (kafka.producer.SyncProducer)
[2012-11-02 06:52:16,412] INFO Disconnecting from host_0998:9092 (kafka.producer.SyncProducer)
[2012-11-02 06:52:18,324] INFO Getting topic metatdata... (kafka.tools.SimpleConsumerShell$)
[2012-11-02 06:52:18,386] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:18,395] ERROR Producer connection to host_0997:9091 unsuccessful (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:18,398] WARN fetching topic metadata for topics [Set(test_1)] from broker [id:0,creatorId:host_0997-1351839138338,host:host_0997,port:9091] failed (kafka.client.ClientUtils$)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:135)
at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:150)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:25)
at kafka.tools.SimpleConsumerShell$.main(SimpleConsumerShell.scala:128)
at kafka.tools.SimpleConsumerShell.main(SimpleConsumerShell.scala)
[2012-11-02 06:52:18,399] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:18,401] INFO Connected to host_0998:9092 for producing (kafka.producer.SyncProducer)
[2012-11-02 06:52:18,446] INFO Disconnecting from host_0998:9092 (kafka.producer.SyncProducer)
[2012-11-02 06:52:20,355] INFO Getting topic metatdata... (kafka.tools.SimpleConsumerShell$)
[2012-11-02 06:52:20,417] INFO Fetching metadata for topic Set(test_1) (kafka.client.ClientUtils$)
[2012-11-02 06:52:20,426] ERROR Producer connection to host_0997:9091 unsuccessful (kafka.producer.SyncProducer)
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
> SimpleConsumerShell ONLY connects to the first host in the broker-list string to fetch topic metadata
> -----------------------------------------------------------------------------------------------------
>
> Key: KAFKA-599
> URL: https://issues.apache.org/jira/browse/KAFKA-599
> Project: Kafka
> Issue Type: Bug
> Reporter: John Fung
> Priority: Critical
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira