You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Hari Shreedharan (JIRA)" <ji...@apache.org> on 2015/02/11 22:43:14 UTC
[jira] [Reopened] (FLUME-2126) Problem in elasticsearch sink when
the event body is a complex field
[ https://issues.apache.org/jira/browse/FLUME-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hari Shreedharan reopened FLUME-2126:
-------------------------------------
Reopening based on above discussion.
> Problem in elasticsearch sink when the event body is a complex field
> --------------------------------------------------------------------
>
> Key: FLUME-2126
> URL: https://issues.apache.org/jira/browse/FLUME-2126
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Environment: 1.3.1 and 1.4
> Reporter: Massimo Paladin
> Assignee: Ashish Paliwal
> Attachments: FLUME-2126-0.patch
>
>
> I have found a bug in the elasticsearch sink, the problem is in the {{ContentBuilderUtil.addComplexField}} method, when it does {{builder.field(fieldName, tmp);}} the {{tmp}} object is taken as {{Object}} with the result of being serialized with the {{toString}} method in the {{XContentBuilder}}. In the end you get the object reference as content.
> The following change workaround the problem for me, the bad point is that it has to parse the content twice, I guess there is a better way to solve the problem but I am not an elasticsearch api expert.
> {code}
> --- a/flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ContentBuilderUtil.java
> +++ b/flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ContentBuilderUtil.java
> @@ -61,7 +61,12 @@ public class ContentBuilderUtil {
> parser = XContentFactory.xContent(contentType).createParser(data);
> parser.nextToken();
> tmp.copyCurrentStructure(parser);
> - builder.field(fieldName, tmp);
> +
> + // if it is a valid structure then we include it
> + parser = XContentFactory.xContent(contentType).createParser(data);
> + parser.nextToken();
> + builder.field(fieldName);
> + builder.copyCurrentStructure(parser);
> } catch (JsonParseException ex) {
> // If we get an exception here the most likely cause is nested JSON that
> // can't be figured out in the body. At this point just push it through
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)