You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Keith Wiley <ke...@atigeo.com> on 2015/07/28 20:33:54 UTC
KafkaConfigurationError: No topics or partitions configured
I'm trying to get a basic consumer off the ground. I can create the consumer but I can't do anything at the message level:
consumer = KafkaConsumer(topic,
group_id=group_id,
bootstrap_servers=[ip + ":" + port])
for m in consumer:
print "x"
Note that I'm not even trying to use the message, I'm just trying loop over the consumer. I'm getting an exception there somehow:
Exception: No topics or partitions configured
Traceback (most recent call last):
File "<timed exec>", line 3, in <module>
File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py", line 290, in next
return six.next(self._get_message_iterator())
File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py", line 324, in fetch_messages
raise KafkaConfigurationError('No topics or partitions configured')
KafkaConfigurationError: No topics or partitions configured
Any ideas? I've been assured (although perhaps incorrectly) that the producer is configured, up and running. Thanks.
Keith Wiley
Senior Software Engineer, Atigeo
keith.wiley@atigeo.com
[atigeo]<http://atigeo.com/>
[twitter]<https://twitter.com/atigeo> [LinkedIn] <https://www.linkedin.com/company/atigeo> [YouTube] <https://www.youtube.com/user/AtigeoXpatterns/> [blog] <http://xpatterns.com/blog/>
Re: KafkaConfigurationError: No topics or partitions configured
Posted by Dana Powers <da...@rd.io>.
Hi Keith,
kafka-python raises FailedPayloadsError on unspecified server failures.
Typically this is caused by a server exception that results in a 0 byte
response. Have you checked your server logs?
-Dana
On Tue, Jul 28, 2015 at 2:01 PM, JIEFU GONG <jg...@berkeley.edu> wrote:
> This won't be very helpful as I am not too experienced with Python or your
> use case, but Java Consumers to indeed have to create a String out of the
> byte array returned from a successful consumption like:
>
> String actualmsg = new String(messageAndMetadata.message())
>
> Consult something like this as it seems you'll need to decode the bytes are
> you are receiving:
> http://stackoverflow.com/questions/606191/convert-bytes-to-a-python-string
>
>
> On Tue, Jul 28, 2015 at 1:41 PM, Keith Wiley <ke...@atigeo.com>
> wrote:
>
> > Thank you. It looks like I had the 'topic' slightly wrong. I didn't
> > realize it was case-sensitive. I got past that error, but now I'm
> bumping
> > up against another error:
> >
> > Traceback (most recent call last):
> > ...
> > File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py",
> > line 59, in __init__
> > self.set_topic_partitions(*topics)
> > File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py",
> > line 242, in set_topic_partitions
> > self._get_commit_offsets()
> > File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py",
> > line 618, in _get_commit_offsets
> > check_error(resp)
> > File "/usr/local/lib/python2.7/dist-packages/kafka/common.py", line
> 230,
> > in check_error
> > raise response
> > kafka.common.FailedPayloadsError
> >
> > I've been told the producer is generating byte array data, not string
> > data. I'm unsure whether that is the cause or what to do about it.
> >
> > Keith Wiley
> > Senior Software Engineer, Atigeo
> > keith.wiley@atigeo.com
> >
> >
> >
> >
> >
> > ________________________________________
> > From: JIEFU GONG <jg...@berkeley.edu>
> > Sent: Tuesday, July 28, 2015 01:31 PM
> > To: users@kafka.apache.org
> > Subject: Re: KafkaConfigurationError: No topics or partitions configured
> >
> > Can you confirm that there are indeed messages in the topic that you
> > published to?
> >
> > bin/kafka-console-consumer.sh --zookeeper [details] --topic [topic]
> > --from-beginning
> >
> > That should be the right command, and you can use that to first verify
> that
> > messages have indeed been published to the topic in question.
> > ᐧ
> >
> > On Tue, Jul 28, 2015 at 11:33 AM, Keith Wiley <ke...@atigeo.com>
> > wrote:
> >
> > > I'm trying to get a basic consumer off the ground. I can create the
> > > consumer but I can't do anything at the message level:
> > >
> > >
> > > consumer = KafkaConsumer(topic,
> > > group_id=group_id,
> > > bootstrap_servers=[ip + ":" + port])
> > >
> > > for m in consumer:
> > > print "x"
> > >
> > > Note that I'm not even trying to use the message, I'm just trying loop
> > > over the consumer. I'm getting an exception there somehow:
> > >
> > >
> > > Exception: No topics or partitions configured
> > > Traceback (most recent call last):
> > > File "<timed exec>", line 3, in <module>
> > > File
> "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py",
> > > line 290, in next
> > > return six.next(self._get_message_iterator())
> > > File
> "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py",
> > > line 324, in fetch_messages
> > > raise KafkaConfigurationError('No topics or partitions configured')
> > > KafkaConfigurationError: No topics or partitions configured
> > >
> > >
> > > Any ideas? I've been assured (although perhaps incorrectly) that the
> > > producer is configured, up and running. Thanks.
> > >
> > >
> > > Keith Wiley
> > > Senior Software Engineer, Atigeo
> > > keith.wiley@atigeo.com
> > >
> > > [atigeo]<http://atigeo.com/>
> > > [twitter]<https://twitter.com/atigeo> [LinkedIn] <
> > > https://www.linkedin.com/company/atigeo> [YouTube] <
> > > https://www.youtube.com/user/AtigeoXpatterns/> [blog] <
> > > http://xpatterns.com/blog/>
> > >
> > >
> >
> >
> > --
> >
> > Jiefu Gong
> > University of California, Berkeley | Class of 2017
> > B.A Computer Science | College of Letters and Sciences
> >
> > jgong@berkeley.edu <el...@berkeley.edu> | (925) 400-3427
> >
>
>
>
> --
>
> Jiefu Gong
> University of California, Berkeley | Class of 2017
> B.A Computer Science | College of Letters and Sciences
>
> jgong@berkeley.edu <el...@berkeley.edu> | (925) 400-3427
>
Re: KafkaConfigurationError: No topics or partitions configured
Posted by JIEFU GONG <jg...@berkeley.edu>.
This won't be very helpful as I am not too experienced with Python or your
use case, but Java Consumers to indeed have to create a String out of the
byte array returned from a successful consumption like:
String actualmsg = new String(messageAndMetadata.message())
Consult something like this as it seems you'll need to decode the bytes are
you are receiving:
http://stackoverflow.com/questions/606191/convert-bytes-to-a-python-string
On Tue, Jul 28, 2015 at 1:41 PM, Keith Wiley <ke...@atigeo.com> wrote:
> Thank you. It looks like I had the 'topic' slightly wrong. I didn't
> realize it was case-sensitive. I got past that error, but now I'm bumping
> up against another error:
>
> Traceback (most recent call last):
> ...
> File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py",
> line 59, in __init__
> self.set_topic_partitions(*topics)
> File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py",
> line 242, in set_topic_partitions
> self._get_commit_offsets()
> File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py",
> line 618, in _get_commit_offsets
> check_error(resp)
> File "/usr/local/lib/python2.7/dist-packages/kafka/common.py", line 230,
> in check_error
> raise response
> kafka.common.FailedPayloadsError
>
> I've been told the producer is generating byte array data, not string
> data. I'm unsure whether that is the cause or what to do about it.
>
> Keith Wiley
> Senior Software Engineer, Atigeo
> keith.wiley@atigeo.com
>
>
>
>
>
> ________________________________________
> From: JIEFU GONG <jg...@berkeley.edu>
> Sent: Tuesday, July 28, 2015 01:31 PM
> To: users@kafka.apache.org
> Subject: Re: KafkaConfigurationError: No topics or partitions configured
>
> Can you confirm that there are indeed messages in the topic that you
> published to?
>
> bin/kafka-console-consumer.sh --zookeeper [details] --topic [topic]
> --from-beginning
>
> That should be the right command, and you can use that to first verify that
> messages have indeed been published to the topic in question.
> ᐧ
>
> On Tue, Jul 28, 2015 at 11:33 AM, Keith Wiley <ke...@atigeo.com>
> wrote:
>
> > I'm trying to get a basic consumer off the ground. I can create the
> > consumer but I can't do anything at the message level:
> >
> >
> > consumer = KafkaConsumer(topic,
> > group_id=group_id,
> > bootstrap_servers=[ip + ":" + port])
> >
> > for m in consumer:
> > print "x"
> >
> > Note that I'm not even trying to use the message, I'm just trying loop
> > over the consumer. I'm getting an exception there somehow:
> >
> >
> > Exception: No topics or partitions configured
> > Traceback (most recent call last):
> > File "<timed exec>", line 3, in <module>
> > File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py",
> > line 290, in next
> > return six.next(self._get_message_iterator())
> > File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py",
> > line 324, in fetch_messages
> > raise KafkaConfigurationError('No topics or partitions configured')
> > KafkaConfigurationError: No topics or partitions configured
> >
> >
> > Any ideas? I've been assured (although perhaps incorrectly) that the
> > producer is configured, up and running. Thanks.
> >
> >
> > Keith Wiley
> > Senior Software Engineer, Atigeo
> > keith.wiley@atigeo.com
> >
> > [atigeo]<http://atigeo.com/>
> > [twitter]<https://twitter.com/atigeo> [LinkedIn] <
> > https://www.linkedin.com/company/atigeo> [YouTube] <
> > https://www.youtube.com/user/AtigeoXpatterns/> [blog] <
> > http://xpatterns.com/blog/>
> >
> >
>
>
> --
>
> Jiefu Gong
> University of California, Berkeley | Class of 2017
> B.A Computer Science | College of Letters and Sciences
>
> jgong@berkeley.edu <el...@berkeley.edu> | (925) 400-3427
>
--
Jiefu Gong
University of California, Berkeley | Class of 2017
B.A Computer Science | College of Letters and Sciences
jgong@berkeley.edu <el...@berkeley.edu> | (925) 400-3427
Re: KafkaConfigurationError: No topics or partitions configured
Posted by Keith Wiley <ke...@atigeo.com>.
Thank you. It looks like I had the 'topic' slightly wrong. I didn't realize it was case-sensitive. I got past that error, but now I'm bumping up against another error:
Traceback (most recent call last):
...
File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py", line 59, in __init__
self.set_topic_partitions(*topics)
File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py", line 242, in set_topic_partitions
self._get_commit_offsets()
File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py", line 618, in _get_commit_offsets
check_error(resp)
File "/usr/local/lib/python2.7/dist-packages/kafka/common.py", line 230, in check_error
raise response
kafka.common.FailedPayloadsError
I've been told the producer is generating byte array data, not string data. I'm unsure whether that is the cause or what to do about it.
Keith Wiley
Senior Software Engineer, Atigeo
keith.wiley@atigeo.com
________________________________________
From: JIEFU GONG <jg...@berkeley.edu>
Sent: Tuesday, July 28, 2015 01:31 PM
To: users@kafka.apache.org
Subject: Re: KafkaConfigurationError: No topics or partitions configured
Can you confirm that there are indeed messages in the topic that you
published to?
bin/kafka-console-consumer.sh --zookeeper [details] --topic [topic]
--from-beginning
That should be the right command, and you can use that to first verify that
messages have indeed been published to the topic in question.
ᐧ
On Tue, Jul 28, 2015 at 11:33 AM, Keith Wiley <ke...@atigeo.com>
wrote:
> I'm trying to get a basic consumer off the ground. I can create the
> consumer but I can't do anything at the message level:
>
>
> consumer = KafkaConsumer(topic,
> group_id=group_id,
> bootstrap_servers=[ip + ":" + port])
>
> for m in consumer:
> print "x"
>
> Note that I'm not even trying to use the message, I'm just trying loop
> over the consumer. I'm getting an exception there somehow:
>
>
> Exception: No topics or partitions configured
> Traceback (most recent call last):
> File "<timed exec>", line 3, in <module>
> File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py",
> line 290, in next
> return six.next(self._get_message_iterator())
> File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py",
> line 324, in fetch_messages
> raise KafkaConfigurationError('No topics or partitions configured')
> KafkaConfigurationError: No topics or partitions configured
>
>
> Any ideas? I've been assured (although perhaps incorrectly) that the
> producer is configured, up and running. Thanks.
>
>
> Keith Wiley
> Senior Software Engineer, Atigeo
> keith.wiley@atigeo.com
>
> [atigeo]<http://atigeo.com/>
> [twitter]<https://twitter.com/atigeo> [LinkedIn] <
> https://www.linkedin.com/company/atigeo> [YouTube] <
> https://www.youtube.com/user/AtigeoXpatterns/> [blog] <
> http://xpatterns.com/blog/>
>
>
--
Jiefu Gong
University of California, Berkeley | Class of 2017
B.A Computer Science | College of Letters and Sciences
jgong@berkeley.edu <el...@berkeley.edu> | (925) 400-3427
Re: KafkaConfigurationError: No topics or partitions configured
Posted by JIEFU GONG <jg...@berkeley.edu>.
Can you confirm that there are indeed messages in the topic that you
published to?
bin/kafka-console-consumer.sh --zookeeper [details] --topic [topic]
--from-beginning
That should be the right command, and you can use that to first verify that
messages have indeed been published to the topic in question.
ᐧ
On Tue, Jul 28, 2015 at 11:33 AM, Keith Wiley <ke...@atigeo.com>
wrote:
> I'm trying to get a basic consumer off the ground. I can create the
> consumer but I can't do anything at the message level:
>
>
> consumer = KafkaConsumer(topic,
> group_id=group_id,
> bootstrap_servers=[ip + ":" + port])
>
> for m in consumer:
> print "x"
>
> Note that I'm not even trying to use the message, I'm just trying loop
> over the consumer. I'm getting an exception there somehow:
>
>
> Exception: No topics or partitions configured
> Traceback (most recent call last):
> File "<timed exec>", line 3, in <module>
> File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py",
> line 290, in next
> return six.next(self._get_message_iterator())
> File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/kafka.py",
> line 324, in fetch_messages
> raise KafkaConfigurationError('No topics or partitions configured')
> KafkaConfigurationError: No topics or partitions configured
>
>
> Any ideas? I've been assured (although perhaps incorrectly) that the
> producer is configured, up and running. Thanks.
>
>
> Keith Wiley
> Senior Software Engineer, Atigeo
> keith.wiley@atigeo.com
>
> [atigeo]<http://atigeo.com/>
> [twitter]<https://twitter.com/atigeo> [LinkedIn] <
> https://www.linkedin.com/company/atigeo> [YouTube] <
> https://www.youtube.com/user/AtigeoXpatterns/> [blog] <
> http://xpatterns.com/blog/>
>
>
--
Jiefu Gong
University of California, Berkeley | Class of 2017
B.A Computer Science | College of Letters and Sciences
jgong@berkeley.edu <el...@berkeley.edu> | (925) 400-3427