You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Mithun Kalan <mi...@gmail.com> on 2015/03/11 13:10:02 UTC
Kafka spout seems act like a bolt
Hi
I've got something weird going on.
[my setup is: I get TCP data using tcpflow, publish to kafka using
kafkacat, storm picks it up with kafkaspout to process and publishes to
redis using jedis. They all run off a single machine with a single
zookeeper]
Storm breaks a few minutes into the run because the messages coming from
kafka do not conform to the format that is expected.
Digging deeper, i found that the culprit messages in kafka look like the
ones in redis.
How is this possible? i never told storm to publish into kafka. Kafka
should never have those messages. I cant find a pattern either. its not a
specific message or a timing issue.
*The lines in kafka should have been:*
2015-02-02 14:03 catergoy: A price: 123
2015-02-02 14:04 catergoy: A price: 124
2015-02-02 14:05 catergoy: A price: 122
*the lines in redis are *
14:04 change +1
14:05 change -2
*but i get this *
2015-02-02 14:03 catergoy: A price: 123
2015-02-02 14:04 catergoy: A price: 124
*3
$7
message
$8
$76
14:04 change +1
2015-02-02 14:05 catergoy: A price: 122
i suspect its a because kafka and storm share the same zookeeper. I will
try to separate them to see if it works.
but has anyone had a similar issue? how did you solve it?
Thank you
Mithun Kalan
Re: Kafka spout seems act like a bolt
Posted by Mithun Kalan <mi...@gmail.com>.
Thanks but not quite
I do not want to publish any Redis related messages to Kafka. But its doing
it anyways. And i dont know how to stop it.
Are you suggesting that i use TCPflow to strip out redis related packets? I
wouldnt know where to begin with this.
On Wed, Mar 11, 2015 at 3:45 PM, 임정택 <ka...@gmail.com> wrote:
> As I understood correctly, you're turning on tcpflow and send a command to
> Redis, right?
> Then maybe Redis protocol could be captured by tcpflow and published to
> Kafka.
> Please check your tcpflow setting.
>
> 2015-03-11 21:10 GMT+09:00 Mithun Kalan <mi...@gmail.com>:
>
>> Hi
>> I've got something weird going on.
>>
>> [my setup is: I get TCP data using tcpflow, publish to kafka using
>> kafkacat, storm picks it up with kafkaspout to process and publishes to
>> redis using jedis. They all run off a single machine with a single
>> zookeeper]
>>
>> Storm breaks a few minutes into the run because the messages coming from
>> kafka do not conform to the format that is expected.
>> Digging deeper, i found that the culprit messages in kafka look like the
>> ones in redis.
>>
>> How is this possible? i never told storm to publish into kafka. Kafka
>> should never have those messages. I cant find a pattern either. its not a
>> specific message or a timing issue.
>>
>> *The lines in kafka should have been:*
>> 2015-02-02 14:03 catergoy: A price: 123
>> 2015-02-02 14:04 catergoy: A price: 124
>> 2015-02-02 14:05 catergoy: A price: 122
>>
>> *the lines in redis are *
>> 14:04 change +1
>> 14:05 change -2
>>
>> *but i get this *
>> 2015-02-02 14:03 catergoy: A price: 123
>> 2015-02-02 14:04 catergoy: A price: 124
>> *3
>> $7
>> message
>> $8
>> $76
>> 14:04 change +1
>> 2015-02-02 14:05 catergoy: A price: 122
>>
>>
>> i suspect its a because kafka and storm share the same zookeeper. I will
>> try to separate them to see if it works.
>> but has anyone had a similar issue? how did you solve it?
>>
>> Thank you
>>
>> Mithun Kalan
>>
>>
>
>
> --
> Name : 임 정택
> Blog : http://www.heartsavior.net / http://dev.heartsavior.net
> Twitter : http://twitter.com/heartsavior
> LinkedIn : http://www.linkedin.com/in/heartsavior
>
--
Mithun Kalan
mithunkalan@gmail.com
Re: Kafka spout seems act like a bolt
Posted by Mithun Kalan <mi...@gmail.com>.
I think fixed it.
Realised i missed "jedis.close();" from my storm bolt. I think that if too
many redis connections are created without being closed, it might be
sending info to a TCP port that happens to be scanned by my TCPflow.
If the problem reappears, ill update the thread.
thanks again
On Wed, Mar 11, 2015 at 3:45 PM, 임정택 <ka...@gmail.com> wrote:
> As I understood correctly, you're turning on tcpflow and send a command to
> Redis, right?
> Then maybe Redis protocol could be captured by tcpflow and published to
> Kafka.
> Please check your tcpflow setting.
>
> 2015-03-11 21:10 GMT+09:00 Mithun Kalan <mi...@gmail.com>:
>
>> Hi
>> I've got something weird going on.
>>
>> [my setup is: I get TCP data using tcpflow, publish to kafka using
>> kafkacat, storm picks it up with kafkaspout to process and publishes to
>> redis using jedis. They all run off a single machine with a single
>> zookeeper]
>>
>> Storm breaks a few minutes into the run because the messages coming from
>> kafka do not conform to the format that is expected.
>> Digging deeper, i found that the culprit messages in kafka look like the
>> ones in redis.
>>
>> How is this possible? i never told storm to publish into kafka. Kafka
>> should never have those messages. I cant find a pattern either. its not a
>> specific message or a timing issue.
>>
>> *The lines in kafka should have been:*
>> 2015-02-02 14:03 catergoy: A price: 123
>> 2015-02-02 14:04 catergoy: A price: 124
>> 2015-02-02 14:05 catergoy: A price: 122
>>
>> *the lines in redis are *
>> 14:04 change +1
>> 14:05 change -2
>>
>> *but i get this *
>> 2015-02-02 14:03 catergoy: A price: 123
>> 2015-02-02 14:04 catergoy: A price: 124
>> *3
>> $7
>> message
>> $8
>> $76
>> 14:04 change +1
>> 2015-02-02 14:05 catergoy: A price: 122
>>
>>
>> i suspect its a because kafka and storm share the same zookeeper. I will
>> try to separate them to see if it works.
>> but has anyone had a similar issue? how did you solve it?
>>
>> Thank you
>>
>> Mithun Kalan
>>
>>
>
>
> --
> Name : 임 정택
> Blog : http://www.heartsavior.net / http://dev.heartsavior.net
> Twitter : http://twitter.com/heartsavior
> LinkedIn : http://www.linkedin.com/in/heartsavior
>
--
Mithun Kalan
mithunkalan@gmail.com
Re: Kafka spout seems act like a bolt
Posted by 임정택 <ka...@gmail.com>.
As I understood correctly, you're turning on tcpflow and send a command to
Redis, right?
Then maybe Redis protocol could be captured by tcpflow and published to
Kafka.
Please check your tcpflow setting.
2015-03-11 21:10 GMT+09:00 Mithun Kalan <mi...@gmail.com>:
> Hi
> I've got something weird going on.
>
> [my setup is: I get TCP data using tcpflow, publish to kafka using
> kafkacat, storm picks it up with kafkaspout to process and publishes to
> redis using jedis. They all run off a single machine with a single
> zookeeper]
>
> Storm breaks a few minutes into the run because the messages coming from
> kafka do not conform to the format that is expected.
> Digging deeper, i found that the culprit messages in kafka look like the
> ones in redis.
>
> How is this possible? i never told storm to publish into kafka. Kafka
> should never have those messages. I cant find a pattern either. its not a
> specific message or a timing issue.
>
> *The lines in kafka should have been:*
> 2015-02-02 14:03 catergoy: A price: 123
> 2015-02-02 14:04 catergoy: A price: 124
> 2015-02-02 14:05 catergoy: A price: 122
>
> *the lines in redis are *
> 14:04 change +1
> 14:05 change -2
>
> *but i get this *
> 2015-02-02 14:03 catergoy: A price: 123
> 2015-02-02 14:04 catergoy: A price: 124
> *3
> $7
> message
> $8
> $76
> 14:04 change +1
> 2015-02-02 14:05 catergoy: A price: 122
>
>
> i suspect its a because kafka and storm share the same zookeeper. I will
> try to separate them to see if it works.
> but has anyone had a similar issue? how did you solve it?
>
> Thank you
>
> Mithun Kalan
>
>
--
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior