[jira] [Comment Edited] (NIFI-1576) Incorporate feedback/improvements to Elasticsearch processors

["Chris Sampson (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17446173#comment-17446173 ] 

Chris Sampson edited comment on NIFI-1576 at 11/18/21, 10:29 PM:
-----------------------------------------------------------------

Implemented point #1 for a new GetElasticsearch processor as part of NIFI-9388 (see [GitHub Pull Request #5535|https://github.com/apache/nifi/pull/5535]).

I'd probably argue that there's little point in expending effort with the Fetch/PutElasticsearch processors that this ticket references now because they were for the very old (and well past EOL) 2.x Elasticsearch clusters. The GetElasticsearch implementation uses the official Elastic connection client and takes a more modern approach (compatible with Elasticsearch 5, 6 & 7, hopefully 8 in the future); that same set of processors already incorporates a PutElasticsearchRecord equivalent to PutElasticsearch along with other useful processors based around the same Elasticsearch connection client API.


was (Author: chris s):
Implemented point #1 for a new GetElasticsearch processor as part of NIFI-9388 (see [GitHub Pull Request #5535|https://github.com/apache/nifi/pull/5535].

I'd probably argue that there's little point in expending effort with the Fetch/PutElasticsearch processors that this ticket references now because they were for the very old (and well past EOL) 2.x Elasticsearch clusters. The GetElasticsearch implementation uses the official Elastic connection client and takes a more modern approach (compatible with Elasticsearch 5, 6 & 7, hopefully 8 in the future); that same set of processors already incorporates a PutElasticsearchRecord equivalent to PutElasticsearch along with other useful processors based around the same Elasticsearch connection client API.

> Incorporate feedback/improvements to Elasticsearch processors
> -------------------------------------------------------------
>
>                 Key: NIFI-1576
>                 URL: https://issues.apache.org/jira/browse/NIFI-1576
>             Project: Apache NiFi
>          Issue Type: Improvement
>            Reporter: Matt Burgess
>            Priority: Major
>
> Since the release of Apache NiFi 0.5.0 containing FetchElasticsearch and PutElasticsearch processors, there have been some improvements identified that would add to the usability, flexibility, and power of the Elasticsearch processor(s). This Jira should capture those improvements for incorporation in a future release.
> Three initial observations:
> 1) FetchElasticsearch currently requires input, but it is possible to configure the processor such that no information is required from an incoming flow file. The processor should be changed to allow (but not require) input.
> 2) FetchElasticsearch specifies a "Document Identifier" property which may or may not contain Expression Language to determine the document ID for fetching. However PutElasticsearch only works when the IDs are unique, so it specifies a "Identifier Attribute" which forces the user to point at an existing attribute. The documentation should be very clear about what the properties are and why/how they're used.
> 3) Related to #2, PutElasticsearch's "Identifier Attribute" is required and a value for that attribute is currently required. Instead, if no value is present, the processor should leverage the auto-generated ID feature of the Elasticsearch API.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)