[jira] [Created] (FLUME-2126) Problem in elasticsearch sink when the event body is a complex field

["Massimo Paladin (JIRA)" <ji...@apache.org>]

Massimo Paladin created FLUME-2126:
--------------------------------------

             Summary: Problem in elasticsearch sink when the event body is a complex field
                 Key: FLUME-2126
                 URL: https://issues.apache.org/jira/browse/FLUME-2126
             Project: Flume
          Issue Type: Bug
          Components: Sinks+Sources
         Environment: 1.3.1 and 1.4
            Reporter: Massimo Paladin


I have found a bug in the elasticsearch sink, the problem is in the {{ContentBuilderUtil.addComplexField}} method, when it does {{builder.field(fieldName, tmp);}} the {{tmp}} object is taken as {{Object}} with the result of being serialized with the {{toString}} method in the {{XContentBuilder}}. In the end you get the object reference as content.

The following change workaround the problem for me, the bad point is that it has to parse the content twice, I guess there is a better way to solve the problem but I am not an elasticsearch api expert. 

{code}
--- a/flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ContentBuilderUtil.java
+++ b/flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ContentBuilderUtil.java
@@ -61,7 +61,12 @@ public class ContentBuilderUtil {
       parser = XContentFactory.xContent(contentType).createParser(data);
       parser.nextToken();
       tmp.copyCurrentStructure(parser);
-      builder.field(fieldName, tmp);
+
+      // if it is a valid structure then we include it
+      parser = XContentFactory.xContent(contentType).createParser(data);
+      parser.nextToken();
+      builder.field(fieldName);
+      builder.copyCurrentStructure(parser);
     } catch (JsonParseException ex) {
       // If we get an exception here the most likely cause is nested JSON that
       // can't be figured out in the body. At this point just push it through
{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira