You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Johny Rufus (JIRA)" <ji...@apache.org> on 2015/05/04 23:50:25 UTC
[jira] [Updated] (FLUME-2220) ElasticSearch sink - duplicate fields
in indexed document
[ https://issues.apache.org/jira/browse/FLUME-2220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Johny Rufus updated FLUME-2220:
-------------------------------
Fix Version/s: (was: v1.6.0)
> ElasticSearch sink - duplicate fields in indexed document
> ---------------------------------------------------------
>
> Key: FLUME-2220
> URL: https://issues.apache.org/jira/browse/FLUME-2220
> Project: Flume
> Issue Type: Bug
> Affects Versions: v1.4.0
> Reporter: Rotem Hermon
> Assignee: Rotem Hermon
> Priority: Minor
> Labels: ElasticSearch, sink
> Attachments: FLUME-2220.patch
>
>
> The default serializer for the ElasticSearch sink (ElasticSearchLogStashEventSerializer) duplicates fields that are mapped to default logstash fields.
> For instance timestamp, source, host. Those appear both as logstash fields ("@timestamp", "@source_host" etc.), and both as fields under the @fields ("@fields.timestamp", "@fields.host").
> When inserting a field from the headers as a logstash system field it should be removed from the dictionary so it wouldn't get written again under the "@fields" field.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)