You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Mike Thomsen (JIRA)" <ji...@apache.org> on 2018/05/16 16:35:00 UTC
[jira] [Commented] (NIFI-5172) PutElasticSearchRecord does to fail
individual recods
[ https://issues.apache.org/jira/browse/NIFI-5172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16477693#comment-16477693 ]
Mike Thomsen commented on NIFI-5172:
------------------------------------
[~mattyb149] AFAIK ES won't tell you which one failed, so if we only fail individual records it would involve hammering ES with 10k - $bad_num individual puts. ES and Solr *really* don't like that unless nothing else is going on with them, particularly WRT ingestion.
> PutElasticSearchRecord does to fail individual recods
> -----------------------------------------------------
>
> Key: NIFI-5172
> URL: https://issues.apache.org/jira/browse/NIFI-5172
> Project: Apache NiFi
> Issue Type: Improvement
> Affects Versions: 1.6.0
> Reporter: Juan C. Sequeiros
> Priority: Minor
>
> My observation and not sure if working as expected but when I send my output from MergeRecord ( set to 10000 ) max number of records and one of those records has an invalid timestamp "bogusdata" value ES rejects it, rightly so since on ES we have a schema template more granular and is expecting timestamp as type "date".
>
> From USER forums Matt Burgess:
> Yes the current behavior is to move the entire input flowfile to
> failure if any errors occur. Some other record-aware processors create
> separate flow files for failed and successful records, but
> PutElasticsearchHttpRecord does not (yet) do that. Please feel free to
> write a Jira for this improvement.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)