You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Felipe Dino <di...@gmail.com> on 2017/04/12 15:32:20 UTC
Interceptor or custom sink?
Hello, I need to develop a solution that consumes a message from a kafka
queue, does a treatment and writes again on another kafka topic.
My question is:
For greater performance, should I treat the message in a custom interceptor
or custom sink?
Grateful.
Felipe Naberezny
Re: [EXT] Interceptor or custom sink?
Posted by Jeff Holoman <jh...@cloudera.com>.
If it's me I write an interceptor and put the events on the Kafka Channel.
You can just do Kafka-Source->Kafka Channel with no sink.
Thanks
Jeff
Re: [EXT] Interceptor or custom sink?
Posted by Shang Wu <Sh...@symantec.com>.
I would choose customer sink since the event could be taken out from the channel after and this eliminates the requirement for the channel space.
PS: Flumeās customer sink is very simple to implement. The following is one of the simplest example
https://github.com/meou/flume-file-sink/blob/master/src/main/java/org/meou/flume/sink/file/TextFileSink.java
BR,
Shang
On 4/12/17, 8:32 AM, "Felipe Dino" <di...@gmail.com> wrote:
Hello, I need to develop a solution that consumes a message from a kafka
queue, does a treatment and writes again on another kafka topic.
My question is:
For greater performance, should I treat the message in a custom interceptor
or custom sink?
Grateful.
Felipe Naberezny