You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by "Gregory M. Foreman" <gf...@spinnerconsulting.com> on 2021/05/06 18:50:05 UTC
JoltTransformRecord producing IllegalTypeConversionException
Hello:
I am using Nifi 1.11.4 and ran into an issue.
JSON Input:
{
"mac":["aabb"]
}
avro schema defined:
{
"type": "record",
"name": "generalInformation",
"namespace": "org.apache.nifi",
"fields": [
{
"name": "mac",
"type": [
"null",
"string",
{"type":"array","items": "string" }
]
}
]
}
No errors experienced when running the file through ConvertRecord. But when using JoltTransformRecord with:
[{
"operation": "shift",
"spec": {
"*": "&"
}
}]
I receive the following:
JoltTransformRecord[id=33f13cf0-fc17-1b9f-9b11-46f210013a8a] Unable to transform StandardFlowFileRecord[uuid=e9905127-b2a2-444a-95bb-baa87523e59d,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1620323949487-603, container=content1, section=603], offset=760454, length=24],offset=0,name=e9905127-b2a2-444a-95bb-baa87523e59d,size=24] due to org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Cannot convert value [[Ljava.lang.Object;@4e63fe90] of type CHOICE[STRING, ARRAY[STRING]] to Map for field mac because the type is not supported: Cannot convert value [[Ljava.lang.Object;@4e63fe90] of type CHOICE[STRING, ARRAY[STRING]] to Map for field mac because the type is not supported
Is there a way to workaround this problem on 1.11.4? This works fine in Nifi 1.12.1.
Thanks,
Greg
Re: JoltTransformRecord producing IllegalTypeConversionException
Posted by "Gregory M. Foreman" <gf...@spinnerconsulting.com>.
Jim:
Thank you for the suggestions. I adjusted the schema to only null or string array and modified the incoming flowfile to cooperate and it worked.
Greg
> On May 6, 2021, at 3:57 PM, Jim Williams <jw...@alertlogic.com> wrote:
>
> Greg -
>
> Maybe use the standard Jolt transformation processor (not the record oriented one) on the initial set of JSON to avoid the need to apply a schema at the start. In that processor, use the 'cardinality' transformation with the MANY option so that any "mac" keys with a scalar string are transformed to an array with a single element.
>
> Then your Avro schema could go like this:
>
> ...
> {
> "name": "mac",
> "type": [
> "null",
> {"type":"array","items": "string" }
> ]
> }
> ...
>
>
> Warm regards,
>
> Jim Williams | Manager, Site Reliability Engineering
>
> jwilliams@alertlogic.com <ma...@alertlogic.com> | alertlogic.com <http://alertlogic.com/>
>
>
>
> -----Original Message-----
> From: Gregory M. Foreman <gf...@spinnerconsulting.com>
> Sent: Thursday, May 6, 2021 1:50 PM
> To: users@nifi.apache.org <ma...@nifi.apache.org>
> Subject: JoltTransformRecord producing IllegalTypeConversionException
>
> Hello:
>
> I am using Nifi 1.11.4 and ran into an issue.
>
> JSON Input:
>
> {
> "mac":["aabb"]
>
> }
>
> avro schema defined:
>
> {
> "type": "record",
> "name": "generalInformation",
> "namespace": "org.apache.nifi",
> "fields": [
> {
> "name": "mac",
> "type": [
> "null",
> "string",
> {"type":"array","items": "string" }
> ]
> }
> ]
> }
>
>
> No errors experienced when running the file through ConvertRecord. But when using JoltTransformRecord with:
>
> [{
> "operation": "shift",
> "spec": {
> "*": "&"
> }
> }]
>
>
> I receive the following:
>
>
> JoltTransformRecord[id=33f13cf0-fc17-1b9f-9b11-46f210013a8a] Unable to transform StandardFlowFileRecord[uuid=e9905127-b2a2-444a-95bb-baa87523e59d,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1620323949487-603, container=content1, section=603], offset=760454, length=24],offset=0,name=e9905127-b2a2-444a-95bb-baa87523e59d,size=24] due to org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Cannot convert value [[Ljava.lang.Object;@4e63fe90] of type CHOICE[STRING, ARRAY[STRING]] to Map for field mac because the type is not supported: Cannot convert value [[Ljava.lang.Object;@4e63fe90] of type CHOICE[STRING, ARRAY[STRING]] to Map for field mac because the type is not supported
>
>
> Is there a way to workaround this problem on 1.11.4? This works fine in Nifi 1.12.1.
>
>
> Thanks,
> Greg
>
>
>
> Confidentiality Notice | This email and any included attachments may be privileged, confidential and/or otherwise protected from disclosure. Access to this email by anyone other than the intended recipient is unauthorized. If you believe you have received this email in error, please contact the sender immediately and delete all copies. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
>
>
> Disclaimer
>
> The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.
>
> This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast, a leader in email security and cyber resilience. Mimecast integrates email defenses with brand protection, security awareness training, web security, compliance and other essential capabilities. Mimecast helps protect large and small organizations from malicious activity, human error and technology failure; and to lead the movement toward building a more resilient world. To find out more, visit our website.
>
RE: JoltTransformRecord producing IllegalTypeConversionException
Posted by Jim Williams <jw...@alertlogic.com>.
Greg -
Maybe use the standard Jolt transformation processor (not the record oriented one) on the initial set of JSON to avoid the need to apply a schema at the start. In that processor, use the 'cardinality' transformation with the MANY option so that any "mac" keys with a scalar string are transformed to an array with a single element.
Then your Avro schema could go like this:
...
{
"name": "mac",
"type": [
"null",
{"type":"array","items": "string" }
]
}
...
Warm regards,
Jim Williams | Manager, Site Reliability Engineering
jwilliams@alertlogic.com | alertlogic.com
-----Original Message-----
From: Gregory M. Foreman <gf...@spinnerconsulting.com>
Sent: Thursday, May 6, 2021 1:50 PM
To: users@nifi.apache.org
Subject: JoltTransformRecord producing IllegalTypeConversionException
Hello:
I am using Nifi 1.11.4 and ran into an issue.
JSON Input:
{
"mac":["aabb"]
}
avro schema defined:
{
"type": "record",
"name": "generalInformation",
"namespace": "org.apache.nifi",
"fields": [
{
"name": "mac",
"type": [
"null",
"string",
{"type":"array","items": "string" }
]
}
]
}
No errors experienced when running the file through ConvertRecord. But when using JoltTransformRecord with:
[{
"operation": "shift",
"spec": {
"*": "&"
}
}]
I receive the following:
JoltTransformRecord[id=33f13cf0-fc17-1b9f-9b11-46f210013a8a] Unable to transform StandardFlowFileRecord[uuid=e9905127-b2a2-444a-95bb-baa87523e59d,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1620323949487-603, container=content1, section=603], offset=760454, length=24],offset=0,name=e9905127-b2a2-444a-95bb-baa87523e59d,size=24] due to org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Cannot convert value [[Ljava.lang.Object;@4e63fe90] of type CHOICE[STRING, ARRAY[STRING]] to Map for field mac because the type is not supported: Cannot convert value [[Ljava.lang.Object;@4e63fe90] of type CHOICE[STRING, ARRAY[STRING]] to Map for field mac because the type is not supported
Is there a way to workaround this problem on 1.11.4? This works fine in Nifi 1.12.1.
Thanks,
Greg
Confidentiality Notice | This email and any included attachments may be privileged, confidential and/or otherwise protected from disclosure. Access to this email by anyone other than the intended recipient is unauthorized. If you believe you have received this email in error, please contact the sender immediately and delete all copies. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
Disclaimer
The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.
This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast, a leader in email security and cyber resilience. Mimecast integrates email defenses with brand protection, security awareness training, web security, compliance and other essential capabilities. Mimecast helps protect large and small organizations from malicious activity, human error and technology failure; and to lead the movement toward building a more resilient world. To find out more, visit our website.