[jira] [Resolved] (NIFI-9890) Record conversion fails in JoltTransformREcord

["Martin Hynar (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-9890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Martin Hynar resolved NIFI-9890.
--------------------------------
    Fix Version/s: 1.20.0
       Resolution: Fixed

The problem is eventually fixed in 1.20.0

> Record conversion fails in JoltTransformREcord
> ----------------------------------------------
>
>                 Key: NIFI-9890
>                 URL: https://issues.apache.org/jira/browse/NIFI-9890
>             Project: Apache NiFi
>          Issue Type: Bug
>    Affects Versions: 1.15.3
>            Reporter: Martin Hynar
>            Priority: Major
>             Fix For: 1.20.0
>
>
> Hello, I am struggling with JOLT transformation of record flow file. The flow is quite simple
>  # Consume records from Kafka - ConsumeKafkaRecord, where reader is Avro Reader Confluent Reference and writer is Json Record Set - this step is OK and flowfiles are correctly sent to success.
>  # Transform records - using JoltTransformRecord, with JsonTreeReader and Json Record Set Writer - the transformation is very simple, new field is created from existing fields. On writing result, bellow exception is thrown. Worth mentioning, the problematic field is not used in the transformation at all.
> The exception is this
>  
> {code:java}
> 2022-04-07 03:32:40,170 ERROR [Timer-Driven Process Thread-1] o.a.n.p.jolt.record.JoltTransformRecord JoltTransformRecord[id=2ea633ad-c1f3-1909-0000-0000686dd600] Unable to transform StandardFlowFileRecord[uuid=e32039eb-e7f9-46ea-a7a4-ad9cf3709b71,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1649063785609-18280, container=default, section=872], offset=381094, length=1143917],offset=0,name=e32039eb-e7f9-46ea-a7a4-ad9cf3709b71,size=1143917] due to org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Cannot convert value [[Ljava.lang.Object;@d786ea4] of type CHOICE[ARRAY[STRING], ARRAY[INT]] to Map for field IgnoreIssues because the type is not supported: org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Cannot convert value [[Ljava.lang.Object;@d786ea4] of type CHOICE[ARRAY[STRING], ARRAY[INT]] to Map for field IgnoreIssues because the type is not supported
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Cannot convert value [[Ljava.lang.Object;@d786ea4] of type CHOICE[ARRAY[STRING], ARRAY[INT]] to Map for field IgnoreIssues because the type is not supported
>     at org.apache.nifi.serialization.record.util.DataTypeUtils.convertRecordFieldtoObject(DataTypeUtils.java:858)
>     at org.apache.nifi.processors.jolt.record.JoltTransformRecord.transform(JoltTransformRecord.java:409)
>     at org.apache.nifi.processors.jolt.record.JoltTransformRecord.onTrigger(JoltTransformRecord.java:334)
>     at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>     at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1273)
>     at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
>     at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:103)
>     at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
>     at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>     at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
>     at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
>     at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>     at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>     at java.base/java.lang.Thread.run(Thread.java:834) {code}
>  
> What I found about the field is that in the record, there are these values
> {noformat}
>   "IgnoreIssues" : [ ],
>   "IgnoreIssues" : [ ],
>   "IgnoreIssues" : [ 0 ],
>   "IgnoreIssues" : [ 0 ],
>   "IgnoreIssues" : [ ],
>   "IgnoreIssues" : [ ],
> {noformat}
> so, single value arrays mixed with empty arrays.
>  
> Also, what I tried was to redirect all failed flow files into SplitRecord processor with max records = 1. When these single record flowfiles were returned to the same transformation, they passed.
> The array is very often empty and I have flow files that pass the transformation, I think it is when all records have empty array in this field.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)