[jira] [Updated] (FLUME-2126) Problem in elasticsearch sink when the event body is a complex field

["Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLUME-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jarek Jarcec Cecho updated FLUME-2126:
--------------------------------------
    Fix Version/s: 1.6

> Problem in elasticsearch sink when the event body is a complex field
> --------------------------------------------------------------------
>
>                 Key: FLUME-2126
>                 URL: https://issues.apache.org/jira/browse/FLUME-2126
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>         Environment: 1.3.1 and 1.4
>            Reporter: Massimo Paladin
>             Fix For: 1.6
>
>         Attachments: FLUME-2126-0.patch
>
>
> I have found a bug in the elasticsearch sink, the problem is in the {{ContentBuilderUtil.addComplexField}} method, when it does {{builder.field(fieldName, tmp);}} the {{tmp}} object is taken as {{Object}} with the result of being serialized with the {{toString}} method in the {{XContentBuilder}}. In the end you get the object reference as content.
> The following change workaround the problem for me, the bad point is that it has to parse the content twice, I guess there is a better way to solve the problem but I am not an elasticsearch api expert. 
> {code}
> --- a/flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ContentBuilderUtil.java
> +++ b/flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ContentBuilderUtil.java
> @@ -61,7 +61,12 @@ public class ContentBuilderUtil {
>        parser = XContentFactory.xContent(contentType).createParser(data);
>        parser.nextToken();
>        tmp.copyCurrentStructure(parser);
> -      builder.field(fieldName, tmp);
> +
> +      // if it is a valid structure then we include it
> +      parser = XContentFactory.xContent(contentType).createParser(data);
> +      parser.nextToken();
> +      builder.field(fieldName);
> +      builder.copyCurrentStructure(parser);
>      } catch (JsonParseException ex) {
>        // If we get an exception here the most likely cause is nested JSON that
>        // can't be figured out in the body. At this point just push it through
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)