[jira] [Resolved] (NIFI-15985) Add per-document Index Field and Timestamp Field extraction to PutElasticsearchJson

["Pierre Villard (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-15985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pierre Villard resolved NIFI-15985.
-----------------------------------
    Fix Version/s: 2.10.0
       Resolution: Fixed

> Add per-document Index Field and Timestamp Field extraction to PutElasticsearchJson
> -----------------------------------------------------------------------------------
>
>                 Key: NIFI-15985
>                 URL: https://issues.apache.org/jira/browse/NIFI-15985
>             Project: Apache NiFi
>          Issue Type: Improvement
>    Affects Versions: 2.9.0
>            Reporter: Adam Turley
>            Assignee: Adam Turley
>            Priority: Major
>             Fix For: 2.10.0
>
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> The Elasticsearch index name can only be set via the Index property using Expression Language over FlowFile attributes. For NDJSON and JSON Array workloads where each document carries its own routing metadata (e.g. an _index or data_stream field), users must add upstream processors to extract and promote those values into FlowFile attributes before reaching PutElasticsearchJson. Similarly, there is no mechanism to map a document field to Elasticsearch's @timestamp field. Additionally, when the existing Identifier Field is used to set the document _id, the source field remains in the document body with no option to remove it.
> Desired Behavior:
> An Index Field property should allow users to specify a field within each document whose value is used as the Elasticsearch index name, falling back to the configured Index property when absent or blank. This should work across all three input formats (NDJSON, JSON Array, Single JSON).
> A Timestamp Field property should allow users to specify a field within each document whose value is written to Elasticsearch as @timestamp, across all three input formats.
> A "Retain Identifier Field", "Retain Index Field", and "Retain Timestamp Field" property should be added for each of the above (including the existing Identifier Field), controlling whether the source field is removed from the document body after extraction. The default should be false (remove the field), since these fields are typically routing or metadata values rather than document content.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)