consume topics from kafka

[Sergio Fernández <wi...@apache.org>]

Hi,

using a KafkaSpout I realized that my topology does not consume the topics
from Kafka (i.e., next time I run the topology the old topics are still
there). Probably it's a misunderstanding from my side, but, how should I
tell my topology that those entries should no remain there once they have
been processed?

Thanks in advance.

Cheers,


-- 
Sergio Fernández
Partner Technology Manager
Redlink GmbH
m: +43 6602747925
e: sergio.fernandez@redlink.co
w: http://redlink.co

Re: consume topics from kafka

["Jake K. Dodd" <ja...@ontopic.io>]

Hi Sergio,

The short answer is that Kafka is a log, not a message queue. When you read a Kafka entry, you're reading from a position in the log--not popping or peeking from a queue.

If you want all the gritty details, I suggest you read this article, and then check out the Kafka docs:

http://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying

http://kafka.apache.org/documentation.html

Best

Jake K Dodd

> On May 28, 2015, at 00:16, Sergio Fernández <wi...@apache.org> wrote:
> 
> Hi,
> 
> using a KafkaSpout I realized that my topology does not consume the topics from Kafka (i.e., next time I run the topology the old topics are still there). Probably it's a misunderstanding from my side, but, how should I tell my topology that those entries should no remain there once they have been processed?
> 
> Thanks in advance.
> 
> Cheers,
> 
> 
> -- 
> Sergio Fernández
> Partner Technology Manager
> Redlink GmbH
> m: +43 6602747925
> e: sergio.fernandez@redlink.co
> w: http://redlink.co