You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Ratha v <vi...@gmail.com> on 2016/04/05 14:13:53 UTC
Is there any behavioural change to connect local server and remote server?
Hi all;
Is there any different connection mechanism for local and remote (ec2
instance) server. Im asking because, my consumer is not working with remote
server.
--
-Ratha
http://vvratha.blogspot.com/
Re: Is there any behavioural change to connect local server and
remote server?
Posted by Ratha v <vi...@gmail.com>.
Here is my current server.properties; (Do I need to change host.name too?)
listeners=PLAINTEXT://:9092
# The port the socket server listens on
port=9092
# Hostname the broker will bind to. If not set, the server will bind to all
interfaces
#host.name=localhost
# Hostname the broker will advertise to producers and consumers. If not
set, it uses the
# value for "host.name" if configured. Otherwise, it will use the value
returned from
# java.net.InetAddress.getCanonicalHostName().
advertised.host.name=kafka.xx.com
# The port to publish to ZooKeeper for clients to use. If this is not set,
# it will publish the same port that the broker binds to.
advertised.port=9092
# The number of threads handling network requests
num.network.threads=3
# The number of threads doing disk I/O
num.io.threads=8
On 6 April 2016 at 10:08, Ratha v <vi...@gmail.com> wrote:
> Hi Ewen;
> Thanks ..Yes broker configuration has been set as you mentioned.
> But when i try following command, i see this exception..Do you know the
> reason?
>
> * #kafka-console-consumer.sh --new-consumer --bootstrap-server
> kafka.xx.com:9092 <http://kafka.xx.com:9092> --topic TEST_NPB4?*
>
> [2016-04-06 09:45:26,855] WARN Error while fetching metadata with
> correlation id 1 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:27,080] WARN Error while fetching metadata with
> correlation id 3 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:27,203] WARN Error while fetching metadata with
> correlation id 5 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:27,326] WARN Error while fetching metadata with
> correlation id 7 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:27,449] WARN Error while fetching metadata with
> correlation id 9 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:27,573] WARN Error while fetching metadata with
> correlation id 11 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:27,699] WARN Error while fetching metadata with
> correlation id 13 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:27,825] WARN Error while fetching metadata with
> correlation id 15 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:27,949] WARN Error while fetching metadata with
> correlation id 17 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:28,075] WARN Error while fetching metadata with
> correlation id 19 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:28,200] WARN Error while fetching metadata with
> correlation id 21 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:28,326] WARN Error while fetching metadata with
> correlation id 23 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:28,450] WARN Error while fetching metadata with
> correlation id 25 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:28,572] WARN Error while fetching metadata with
> correlation id 27 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:28,696] WARN Error while fetching metadata with
> correlation id 29 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:28,820] WARN Error while fetching metadata with
> correlation id 31 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:28,945] WARN Error while fetching metadata with
> correlation id 33 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
> (org.apache.kafka.clients.NetworkClient)
>
> [2016-04-06 09:45:29,070]
>
> On 6 April 2016 at 07:09, Ewen Cheslack-Postava <ew...@confluent.io> wrote:
>
>> Ratha,
>>
>> In EC2, you probably need to use the advertised.listeners setting (or
>> advertised.host and advertised.port on older brokers). This is because EC2
>> has internal and external addresses for each instance.
>>
>> -Ewen
>>
>> On Tue, Apr 5, 2016 at 5:13 AM, Ratha v <vi...@gmail.com> wrote:
>>
>> > Hi all;
>> > Is there any different connection mechanism for local and remote (ec2
>> > instance) server. Im asking because, my consumer is not working with
>> remote
>> > server.
>> > --
>> > -Ratha
>> > http://vvratha.blogspot.com/
>> >
>>
>>
>>
>> --
>> Thanks,
>> Ewen
>>
>
>
>
> --
> -Ratha
> http://vvratha.blogspot.com/
>
--
-Ratha
http://vvratha.blogspot.com/
Re: Is there any behavioural change to connect local server and
remote server?
Posted by Ratha v <vi...@gmail.com>.
Hi Ewen;
Thanks ..Yes broker configuration has been set as you mentioned.
But when i try following command, i see this exception..Do you know the
reason?
* #kafka-console-consumer.sh --new-consumer --bootstrap-server
kafka.xx.com:9092 <http://kafka.xx.com:9092> --topic TEST_NPB4?*
[2016-04-06 09:45:26,855] WARN Error while fetching metadata with
correlation id 1 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:27,080] WARN Error while fetching metadata with
correlation id 3 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:27,203] WARN Error while fetching metadata with
correlation id 5 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:27,326] WARN Error while fetching metadata with
correlation id 7 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:27,449] WARN Error while fetching metadata with
correlation id 9 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:27,573] WARN Error while fetching metadata with
correlation id 11 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:27,699] WARN Error while fetching metadata with
correlation id 13 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:27,825] WARN Error while fetching metadata with
correlation id 15 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:27,949] WARN Error while fetching metadata with
correlation id 17 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:28,075] WARN Error while fetching metadata with
correlation id 19 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:28,200] WARN Error while fetching metadata with
correlation id 21 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:28,326] WARN Error while fetching metadata with
correlation id 23 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:28,450] WARN Error while fetching metadata with
correlation id 25 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:28,572] WARN Error while fetching metadata with
correlation id 27 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:28,696] WARN Error while fetching metadata with
correlation id 29 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:28,820] WARN Error while fetching metadata with
correlation id 31 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:28,945] WARN Error while fetching metadata with
correlation id 33 : {TEST_NPB4?=INVALID_TOPIC_EXCEPTION}
(org.apache.kafka.clients.NetworkClient)
[2016-04-06 09:45:29,070]
On 6 April 2016 at 07:09, Ewen Cheslack-Postava <ew...@confluent.io> wrote:
> Ratha,
>
> In EC2, you probably need to use the advertised.listeners setting (or
> advertised.host and advertised.port on older brokers). This is because EC2
> has internal and external addresses for each instance.
>
> -Ewen
>
> On Tue, Apr 5, 2016 at 5:13 AM, Ratha v <vi...@gmail.com> wrote:
>
> > Hi all;
> > Is there any different connection mechanism for local and remote (ec2
> > instance) server. Im asking because, my consumer is not working with
> remote
> > server.
> > --
> > -Ratha
> > http://vvratha.blogspot.com/
> >
>
>
>
> --
> Thanks,
> Ewen
>
--
-Ratha
http://vvratha.blogspot.com/
Re: Is there any behavioural change to connect local server and
remote server?
Posted by Ewen Cheslack-Postava <ew...@confluent.io>.
Ratha,
In EC2, you probably need to use the advertised.listeners setting (or
advertised.host and advertised.port on older brokers). This is because EC2
has internal and external addresses for each instance.
-Ewen
On Tue, Apr 5, 2016 at 5:13 AM, Ratha v <vi...@gmail.com> wrote:
> Hi all;
> Is there any different connection mechanism for local and remote (ec2
> instance) server. Im asking because, my consumer is not working with remote
> server.
> --
> -Ratha
> http://vvratha.blogspot.com/
>
--
Thanks,
Ewen