You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Alexander Denissov (Jira)" <ji...@apache.org> on 2020/09/17 22:08:00 UTC
[jira] [Updated] (NIFI-7817) ParquetReader fails to instantiate due
to missing default value for compression-type property
[ https://issues.apache.org/jira/browse/NIFI-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Denissov updated NIFI-7817:
-------------------------------------
Description:
We have a customer Processor that uses ParquetReader to parse incoming Flow File and then send records to the target system. It worked before, but broke with NiFi 1.12.0 release.
The exception stack trace:
{{{{Caused by: java.lang.NullPointerException: Name is null}}}}
{{ at java.lang.Enum.valueOf(Enum.java:236)}}
{{ at org.apache.parquet.hadoop.metadata.CompressionCodecName.valueOf(CompressionCodecName.java:26)}}
{{ at org.apache.nifi.parquet.utils.ParquetUtils.createParquetConfig(ParquetUtils.java:172)}}
{{ at org.apache.nifi.parquet.ParquetReader.createRecordReader(ParquetReader.java:48)}}
{{ at org.apache.nifi.serialization.RecordReaderFactory.createRecordReader(RecordReaderFactory.java:49)}}
{{ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)}}
{{ at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
{{ at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
{{ at java.lang.reflect.Method.invoke(Method.java:498)}}
{{ at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)}}
{{ at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:105)}}
{{ at com.sun.proxy.$Proxy100.createRecordReader(Unknown Source)}}
{{ at io.pivotal.greenplum.nifi.processors.PutGreenplumRecord.lambda$null$1(PutGreenplumRecord.java:300)}}
{{ at org.apache.nifi.processor.util.pattern.ExceptionHandler.execute(ExceptionHandler.java:127)}}
Basically, the creation of a ParquetReader by the RecordReaderFactory fails.
The actual problem occurs here: [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-parquet-bundle/nifi-parquet-processors/src/main/java/org/apache/nifi/parquet/utils/ParquetUtils.java#L171-L172]
where
{{final String compressionTypeValue = context.getProperty(ParquetUtils.COMPRESSION_TYPE).getValue();}}
comes back with the value of null, since "compression-type" property is not exposed on ParquetReader and would not be set by the flow designers. The returned null value is then passed to get the enum instance and fails there.
{{final CompressionCodecName codecName = CompressionCodecName.valueOf(compressionTypeValue);}}
While there might be several solutions to this, including updating parquet-specific defaulting logic, I traced the root cause of this regression to the fix for NIFI-7635, to this commit: [https://github.com/apache/nifi/commit/4f11e3626093d3090f97c0efc5e229d83b6006e4#diff-782335ecee68f6939c3724dba3983d3d]
where the default value of provided property descriptor, expressed previously as
property.getDefaultValue()
is no longer used. That value used to contain UNCOMPRESSED value for the use case in question and it used to work before this commit.
I'd think the issue needs to get fixed in this place as it might affect a variety of other use cases.
was:
We have a customer Processor that uses ParquetReader to parse incoming Flow File and then send records to the target system. It worked before, but broke with NiFi 1.12.0 release.
The exception stack trace:
```
{{Caused by: java.lang.NullPointerException: Name is null}}
{{ at java.lang.Enum.valueOf(Enum.java:236)}}
{{ at org.apache.parquet.hadoop.metadata.CompressionCodecName.valueOf(CompressionCodecName.java:26)}}
{{ at org.apache.nifi.parquet.utils.ParquetUtils.createParquetConfig(ParquetUtils.java:172)}}
{{ at org.apache.nifi.parquet.ParquetReader.createRecordReader(ParquetReader.java:48)}}
{{ at org.apache.nifi.serialization.RecordReaderFactory.createRecordReader(RecordReaderFactory.java:49)}}
{{ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)}}
{{ at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
{{ at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
{{ at java.lang.reflect.Method.invoke(Method.java:498)}}
{{ at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)}}
{{ at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:105)}}
{{ at com.sun.proxy.$Proxy100.createRecordReader(Unknown Source)}}
{{ at io.pivotal.greenplum.nifi.processors.PutGreenplumRecord.lambda$null$1(PutGreenplumRecord.java:300)}}
{{ at org.apache.nifi.processor.util.pattern.ExceptionHandler.execute(ExceptionHandler.java:127)}}
```
Basically, the creation of a ParquetReader by the RecordReaderFactory fails.
The actual problem occurs here: [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-parquet-bundle/nifi-parquet-processors/src/main/java/org/apache/nifi/parquet/utils/ParquetUtils.java#L171-L172]
where
{{final String compressionTypeValue = context.getProperty(ParquetUtils.COMPRESSION_TYPE).getValue();}}
comes back with the value of null, since "compression-type" property is not exposed on ParquetReader and would not be set by the flow designers. The returned null value is then passed to get the enum instance and fails there.
{{final CompressionCodecName codecName = CompressionCodecName.valueOf(compressionTypeValue);}}
While there might be several solutions to this, including updating parquet-specific defaulting logic, I traced the root cause of this regression to the fix for NIFI-7635, to this commit: [https://github.com/apache/nifi/commit/4f11e3626093d3090f97c0efc5e229d83b6006e4#diff-782335ecee68f6939c3724dba3983d3d]
where the default value of provided property descriptor, expressed previously as
property.getDefaultValue()
is no longer used. That value used to contain UNCOMPRESSED value for the use case in question and it used to work before this commit.
I'd think the issue needs to get fixed in this place as it might affect a variety of other use cases.
> ParquetReader fails to instantiate due to missing default value for compression-type property
> ---------------------------------------------------------------------------------------------
>
> Key: NIFI-7817
> URL: https://issues.apache.org/jira/browse/NIFI-7817
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.12.0
> Reporter: Alexander Denissov
> Priority: Major
>
> We have a customer Processor that uses ParquetReader to parse incoming Flow File and then send records to the target system. It worked before, but broke with NiFi 1.12.0 release.
> The exception stack trace:
> {{{{Caused by: java.lang.NullPointerException: Name is null}}}}
> {{ at java.lang.Enum.valueOf(Enum.java:236)}}
> {{ at org.apache.parquet.hadoop.metadata.CompressionCodecName.valueOf(CompressionCodecName.java:26)}}
> {{ at org.apache.nifi.parquet.utils.ParquetUtils.createParquetConfig(ParquetUtils.java:172)}}
> {{ at org.apache.nifi.parquet.ParquetReader.createRecordReader(ParquetReader.java:48)}}
> {{ at org.apache.nifi.serialization.RecordReaderFactory.createRecordReader(RecordReaderFactory.java:49)}}
> {{ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)}}
> {{ at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
> {{ at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
> {{ at java.lang.reflect.Method.invoke(Method.java:498)}}
> {{ at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)}}
> {{ at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:105)}}
> {{ at com.sun.proxy.$Proxy100.createRecordReader(Unknown Source)}}
> {{ at io.pivotal.greenplum.nifi.processors.PutGreenplumRecord.lambda$null$1(PutGreenplumRecord.java:300)}}
> {{ at org.apache.nifi.processor.util.pattern.ExceptionHandler.execute(ExceptionHandler.java:127)}}
>
> Basically, the creation of a ParquetReader by the RecordReaderFactory fails.
> The actual problem occurs here: [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-parquet-bundle/nifi-parquet-processors/src/main/java/org/apache/nifi/parquet/utils/ParquetUtils.java#L171-L172]
> where
> {{final String compressionTypeValue = context.getProperty(ParquetUtils.COMPRESSION_TYPE).getValue();}}
> comes back with the value of null, since "compression-type" property is not exposed on ParquetReader and would not be set by the flow designers. The returned null value is then passed to get the enum instance and fails there.
> {{final CompressionCodecName codecName = CompressionCodecName.valueOf(compressionTypeValue);}}
> While there might be several solutions to this, including updating parquet-specific defaulting logic, I traced the root cause of this regression to the fix for NIFI-7635, to this commit: [https://github.com/apache/nifi/commit/4f11e3626093d3090f97c0efc5e229d83b6006e4#diff-782335ecee68f6939c3724dba3983d3d]
> where the default value of provided property descriptor, expressed previously as
> property.getDefaultValue()
> is no longer used. That value used to contain UNCOMPRESSED value for the use case in question and it used to work before this commit.
> I'd think the issue needs to get fixed in this place as it might affect a variety of other use cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)