You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Alex Tian (JIRA)" <ji...@apache.org> on 2015/09/09 17:07:46 UTC
[jira] [Updated] (FLUME-2789) Kafka sink sends messages to the
source topic by mistake
[ https://issues.apache.org/jira/browse/FLUME-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Tian updated FLUME-2789:
-----------------------------
Description:
In my scenario, I need to send messages from a kafka source to a kafka sink , in other workds, transfering messages from a topic A to another topic B.
As a result, I find my kafka sink always sends messages back to topic A.
The reason in the codes is:
KafkaSink.java:
****************************************************************************************
107 if ((eventTopic = headers.get(TOPIC_HDR)) == null) {
108 eventTopic = topic;
109 }
If the source is a Kafka source, eventTopic won't be null because it is the source topic from the msg header. Therefore, the kafka sink will send the message to its source topic by mistake.
I think we should remove line 107 and line 109 (Making "eventTopic = topic" anyway)
was:
In my scenario, I need to send messages from a kafka source to a kafka sink , in other workds, transfering messages from a topic A to another topic B.
As a result, I find my kafka sink always sends messages back to topic A.
The reason in the codes is:
KafkaSink.java:
****************************************************************************************
107 if ((eventTopic = headers.get(TOPIC_HDR)) == null) {
108 eventTopic = topic;
109 }
If the source is a Kafka source, eventTopic won't be null because it is the source topic from the msg header. Therefore, the kafka sink will send the message to its source topic by mistake.
I think we should remove line 107 and line 109 (Making "eventTopic = topic" anyway)
> Kafka sink sends messages to the source topic by mistake
> ---------------------------------------------------------
>
> Key: FLUME-2789
> URL: https://issues.apache.org/jira/browse/FLUME-2789
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.6.0
> Environment: Flume 1.6
> Reporter: Alex Tian
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> In my scenario, I need to send messages from a kafka source to a kafka sink , in other workds, transfering messages from a topic A to another topic B.
> As a result, I find my kafka sink always sends messages back to topic A.
> The reason in the codes is:
> KafkaSink.java:
> ****************************************************************************************
> 107 if ((eventTopic = headers.get(TOPIC_HDR)) == null) {
> 108 eventTopic = topic;
> 109 }
> If the source is a Kafka source, eventTopic won't be null because it is the source topic from the msg header. Therefore, the kafka sink will send the message to its source topic by mistake.
> I think we should remove line 107 and line 109 (Making "eventTopic = topic" anyway)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)