You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "endzeit (Jira)" <ji...@apache.org> on 2024/08/08 04:12:00 UTC
[jira] [Updated] (NIFI-13468) Add RecordPath function recordOf
[ https://issues.apache.org/jira/browse/NIFI-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
endzeit updated NIFI-13468:
---------------------------
Status: Patch Available (was: Open)
> Add RecordPath function recordOf
> --------------------------------
>
> Key: NIFI-13468
> URL: https://issues.apache.org/jira/browse/NIFI-13468
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: endzeit
> Assignee: endzeit
> Priority: Major
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> NiFi version 1.25.0 / NIFI-12538 introduced a new standalone function to the RecordPath DSL named {{{}mapOf{}}}.
> As the name suggests, it allows DSL users to create more complex data structures, namely maps, using the DSL.
> However, due to restrictions in the Record data type definitions, it only supports the creation of map structures whose values are all of the same type.
> The current implementation even explicitly limits the values to be of type {{{}String{}}}, effectively limiting the result type of {{mapOf}} to {{Map<String, String>}} but this limit could be lifted in the future to support {{Map<String, T>}} instead.
> Due to the underlying restriction enforced by the {{{}org.apache.nifi.serialization.record.type.MapDataType{}}}, adding support for differently typed values seems not feasible. Thus this issue proposes the addition of a {{recordOf}} standalone function to the RecordPath DSL, that allows DSL users to create data structures of type {{org.apache.nifi.serialization.record.type.RecordDataType}} as well.
> {{recordOf}} has the same requirements about its arguments, namely
> - there has be an even number of arguments provided
> - every odd argument must resemble a String used as field name
> - every even argument is used as field value
>
> ----
> _An example_
> As {{mapOf}} and {{recordOf}} might look similar from the outside, lets present a behavioral difference using an example. I'll make use of the existing {{encodeJson}} RecordPath DSL function to turn the result into JSON as it makes the differences quite obvious.
> Assume we have the following record with different types of fields.
> {code:json}
> {
> "aLong": 9876543210,
> "aDouble": 2.5,
> "aString": "texty",
> "anArray": [
> "a",
> "b",
> "c"
> ],
> "aMap": {
> "anotherKey": "anotherValue",
> "aKey": "aValue"
> },
> "aRecord": {
> "aField": 2,
> "anotherField": "aRecordValue"
> }
> }
> {code}
> The result of {{recordOf}} preserves those types.
> {noformat}
> escapeJson(recordOf('mappedLong', /aLong, 'mappedDouble', /aDouble, 'mappedString', /aString, 'mappedArray', /anArray, 'mappedMap', /aMap, 'mappedRecord', /aRecord)){noformat}
> {code:json}
> {
> "mappedLong": 9876543210,
> "mappedDouble": 2.5,
> "mappedString": "texty",
> "mappedArray": [
> "a",
> "b",
> "c"
> ],
> "mappedMap": {
> "anotherKey": "anotherValue",
> "aKey": "aValue"
> },
> "mappedRecord": {
> "aField": 2,
> "anotherField": "aRecordValue"
> }
> }
> {code}
> With {{{}mapOf{}}}, all types are coerced to String instead.
> {noformat}
> escapeJson(mapOf('mappedLong', /aLong, 'mappedDouble', /aDouble, 'mappedString', /aString, 'mappedArray', /anArray, 'mappedMap', /aMap, 'mappedRecord', /aRecord)){noformat}
> {code:json}
> {
> "mappedLong": "9876543210",
> "mappedDouble": "2.5",
> "mappedString": "texty",
> "mappedArray": "[a, b, c]",
> "mappedMap": "{anotherKey=anotherValue, aKey=aValue}",
> "mappedRecord": "MapRecord[{anotherField=aRecordValue, aField=2}]"
> }
> {code}
>
>
> ----
> _Why not augment {{mapOf}} instead?_
> I thought about this. Actually this was my initial idea.
> However, {{mapOf}} has been introduced almost 6 months ago. Adding support for different types of values is a backwards incompatible change. It may break existing code that expects values being coerced into {{String}} as its done at the moment.
> Additionally, it may seems strange that a function called "mapOf" actually creates a "Record" instead of a "Map".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)