[jira] [Updated] (NIFI-7197) In-place replacement in LookupRecord processor

["Pierre Villard (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pierre Villard updated NIFI-7197:
---------------------------------
    Status: Patch Available  (was: Open)

> In-place replacement in LookupRecord processor
> ----------------------------------------------
>
>                 Key: NIFI-7197
>                 URL: https://issues.apache.org/jira/browse/NIFI-7197
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>            Reporter: Pierre Villard
>            Assignee: Pierre Villard
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> One of the promise of the record-oriented processors is to avoid any data splitting to increase overall performances. This JIRA is about improving the {{LookupRecord}} processor to support lookup replacement when the lookup key field is located into an array of the input data.
> Here is an example of data:
> {code:java}
>   {
>     "locales": [
>       {
>         "language" : "fr",
>         "region" : "CA"
>       },
>       {
>         "language" : "fr",
>         "region" : "FR"
>       }
>     ]
>   }
> {code}
> If we want to replace language for each occurrence in the array, we'd need to slip the data, do the lookup and then merge back everything together.
> Unfortunately we can't use something like:
> {code:java}
> Result RecordPath: /locales[*]/language
> key: /locales[*]/language
> {code}
> Because the provided result record path returns multiple fields.
> With this JIRA, I suggest to add a new boolean property for "in-place replacement" that would default to false to preserve current behavior. It would also allow to do multi replacements at one time.
> If true, the following requirements would be made:
> - the lookup service should be a single key lookup service
> - replacement will only support simple types
> Then, every dynamic property containing record path would be evaluated and the corresponding field will be used as the key with the lookup service and the field will be updated with the value returned by the lookup service.
> Example:
> {code:java}
> [
>   {
>     "foo" : {
>       "foo" : "key"
>     },
>     "locales": [
>       {
>         "language" : "fr",
>         "region" : "CA"
>       },
>       {
>         "language" : "fr",
>         "region" : "FR"
>       }
>     ]
>   }, 
>     {
>       "foo" : {
>         "foo" : "key"
>       },
>       "locales": [
>         {
>           "language" : "fr",
>           "region" : "CA"
>         },
>         {
>           "language" : "fr",
>           "region" : "FR"
>         }
>       ]
>     }
> ]
> {code}
> With a lookup service containing:
> {code:java}
> fr => French
> CA => Canada
> FR => France
> key => value
> {code}
> And the following dynamic properties:
> {code:java}
> - lookupLanguage => /locales[*]/language
> - lookupRegion => /locales[*]/region
> - lookupFoo => /foo/foo
> {code}
> should give
> {code:java}
> [
>   {
>     "foo" : {
>       "foo" : "value"
>     },
>     "locales": [
>       {
>         "language" : "French",
>         "region" : "Canada"
>       },
>       {
>         "language" : "French",
>         "region" : "France"
>       }
>     ]
>   },  {
>       "foo" : {
>         "foo" : "value"
>       },
>       "locales": [
>         {
>           "language" : "French",
>           "region" : "Canada"
>         },
>         {
>           "language" : "French",
>           "region" : "France"
>         }
>       ]
>     }
> ]
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)