You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Vibhath Ileperuma <vi...@gmail.com> on 2022/08/02 05:54:15 UTC
Re: Multiplying a record filed value by a constant
Hi Nathan,
The value of the field can be an Integer or it can be empty. Due to that, I
have configured that field as a nullable field in the schema registry.
When I tried to stamp 0 when the value is null using belo Expression, it
works properly. Hence I believe error is not coming from the reader.
*${field.value:isNull():ifElse(0 , ${field.value:multiply(1000000)})}*
When I check the nifi-app.log there is a number format exception when the
value is empty.
*2022-08-02 06:26:30,850 ERROR [Timer-Driven Process Thread-11]
o.a.n.processors.standard.UpdateRecord
UpdateRecord[id=01821014-6ce9-14b9-4604-6eea727a933b] Failed to process
StandardFlowFileRecord[uuid=dcd98ea2-f0f9-4bce-a139-9633980dd078,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1659417986035-1, container=default,
section=1], offset=0,
length=23202895],offset=0,name=export_orders_including_system_events.csv,size=23202895];
will route to failure: java.lang.NumberFormatException: For input string:
""java.lang.NumberFormatException: For input string: "" at
java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.base/java.lang.Long.parseLong(Long.java:702) at
java.base/java.lang.Long.parseLong(Long.java:817) at
org.apache.nifi.serialization.record.util.DataTypeUtils.toLong(DataTypeUtils.java:1521)
at
org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:202)
at
org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:153)
at
org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:149)
at
org.apache.nifi.serialization.record.MapRecord.setValueAndGetField(MapRecord.java:362)
at
org.apache.nifi.serialization.record.MapRecord.setValue(MapRecord.java:320)
at
org.apache.nifi.record.path.StandardFieldValue.updateValue(StandardFieldValue.java:144)
at
org.apache.nifi.record.path.StandardFieldValue.updateValue(StandardFieldValue.java:128)
at
org.apache.nifi.processors.standard.UpdateRecord.lambda$process$0(UpdateRecord.java:185)
at
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at
java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at
java.base/java.util.stream.Streams$StreamBuilderImpl.forEachRemaining(Streams.java:411)
at
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at
java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at
java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at
java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
at
org.apache.nifi.processors.standard.UpdateRecord.process(UpdateRecord.java:177)
at
org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:158)
at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:3158)*
Seems the error is occuring when the results are going to be written as a
long.
Hence I believe an 'if' function is required for expression language in
addition to the 'IfElse' function to overcome this kind of scenario.
Thank You.
Best Regards,
Vibhath.
On Sun, Jul 31, 2022 at 1:31 PM nathan.english.bt.com via users <
users@nifi.apache.org> wrote:
> Hi Vibhath,
>
> I could be completely wrong here. Apologies in Advanced if I am!
>
> What is the value of the field before the Update Record processor? It
> seems like it's an empty string? How do you have the CSV Reader Configured?
>
> Is this reader used anywhere else in the flow? If so, does it error, there
> too?
>
> Based on the limited logging shared, I don't think the Update Record has
> even got round to working on the field, I think the CSV Reader hasn't even
> been able to read the file correctly and is erroring.
>
> Out of Curiosity, Have you tried setting the default value to null on the
> reader schema?
>
> Are there any further logs for this error, if so, are you able to share?
>
> Kind Regards,
>
> Nathan
>
> Get Outlook for iOS <https://aka.ms/o0ukef>
> ------------------------------
> *From:* Vibhath Ileperuma <vi...@gmail.com>
> *Sent:* Sunday, July 31, 2022 8:49:04 AM
> *To:* users@nifi.apache.org <us...@nifi.apache.org>
> *Subject:* Multiplying a record filed value by a constant
>
> Hi All,
>
> I'm using Nifi's expression language with 'Update Record' processor to
> multiply an integer value in a record path by a constant. Note that my
> input is a csv file and I'm providing the schema through a Avro schema
> registry. This filed I'm using is specified as a nullable filed in the
> schema as below and this filed is null in some records.
>
>
>
>
>
>
> * { "type": [ "long", "null"
> ], "name": "Insert_Timestamp" }*
>
> What I want do is to multiply the filed value when it is not null and do
> nothing when it is null.
>
> When I simply multiply the value as below, it throws an error when the
> value is null.
> *${field.value:multiply(1000000)}*
>
> Hence I tried to use a null check and a IfElse before multiplying as below.
> *${field.value:isNull():ifElse(${field.value} ,
> ${field.value:multiply(1000000)})}*
>
> Then also I'm getting an error when the value is null.
>
> *UpdateRecord[id=01821014-6ce9-14b9-4604-6eea727a933b] Failed to process StandardFlowFileRecord[uuid=abd624d4-d571-49c8-9b1a-5e171b487e33,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1659246191387-1, container=default, section=1], offset=0, length=23202895],offset=0,name=abc.csv,size=23202895]; will route to failure: For input string: ""*
>
>
> Could you please let me know whether there is a way to mutiply the record only if it is not null.
>
> Thank You.
>
> Best Regards,
>
> Vibhath Ileperuma.
>
Re: Multiplying a record filed value by a constant
Posted by Cannon Palms <cp...@influxdata.com>.
You're checking for `isNull`, but this value is not null, it is empty. Can
you add `isEmpty` as well before Multiplying? I also wonder if this value
is always a numeric string, rather than a number. If so, you'll also need
`toNumber`.
Try this, and consider adding toNumber if the field is actually a string
and not an integer.
${field.value:isNull():ifElse(0, ${field.value:isEmpty():ifElse(0,
${field.value:multiply(1000000)})})}
Cannon
On Tue, Aug 2, 2022, 1:54 AM Vibhath Ileperuma <vi...@gmail.com>
wrote:
> Hi Nathan,
>
> The value of the field can be an Integer or it can be empty. Due to that,
> I have configured that field as a nullable field in the schema registry.
>
> When I tried to stamp 0 when the value is null using belo Expression, it
> works properly. Hence I believe error is not coming from the reader.
> *${field.value:isNull():ifElse(0 , ${field.value:multiply(1000000)})}*
>
> When I check the nifi-app.log there is a number format exception when the
> value is empty.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *2022-08-02 06:26:30,850 ERROR [Timer-Driven Process Thread-11]
> o.a.n.processors.standard.UpdateRecord
> UpdateRecord[id=01821014-6ce9-14b9-4604-6eea727a933b] Failed to process
> StandardFlowFileRecord[uuid=dcd98ea2-f0f9-4bce-a139-9633980dd078,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1659417986035-1, container=default,
> section=1], offset=0,
> length=23202895],offset=0,name=export_orders_including_system_events.csv,size=23202895];
> will route to failure: java.lang.NumberFormatException: For input string:
> ""java.lang.NumberFormatException: For input string: "" at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:702) at
> java.base/java.lang.Long.parseLong(Long.java:817) at
> org.apache.nifi.serialization.record.util.DataTypeUtils.toLong(DataTypeUtils.java:1521)
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:202)
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:153)
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:149)
> at
> org.apache.nifi.serialization.record.MapRecord.setValueAndGetField(MapRecord.java:362)
> at
> org.apache.nifi.serialization.record.MapRecord.setValue(MapRecord.java:320)
> at
> org.apache.nifi.record.path.StandardFieldValue.updateValue(StandardFieldValue.java:144)
> at
> org.apache.nifi.record.path.StandardFieldValue.updateValue(StandardFieldValue.java:128)
> at
> org.apache.nifi.processors.standard.UpdateRecord.lambda$process$0(UpdateRecord.java:185)
> at
> java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at
> java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
> at
> java.base/java.util.stream.Streams$StreamBuilderImpl.forEachRemaining(Streams.java:411)
> at
> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
> at
> java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
> at
> java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at
> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
> at
> org.apache.nifi.processors.standard.UpdateRecord.process(UpdateRecord.java:177)
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:158)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:3158)*
>
> Seems the error is occuring when the results are going to be written as a
> long.
> Hence I believe an 'if' function is required for expression language in
> addition to the 'IfElse' function to overcome this kind of scenario.
>
> Thank You.
> Best Regards,
> Vibhath.
>
>
> On Sun, Jul 31, 2022 at 1:31 PM nathan.english.bt.com via users <
> users@nifi.apache.org> wrote:
>
>> Hi Vibhath,
>>
>> I could be completely wrong here. Apologies in Advanced if I am!
>>
>> What is the value of the field before the Update Record processor? It
>> seems like it's an empty string? How do you have the CSV Reader Configured?
>>
>> Is this reader used anywhere else in the flow? If so, does it error,
>> there too?
>>
>> Based on the limited logging shared, I don't think the Update Record has
>> even got round to working on the field, I think the CSV Reader hasn't even
>> been able to read the file correctly and is erroring.
>>
>> Out of Curiosity, Have you tried setting the default value to null on the
>> reader schema?
>>
>> Are there any further logs for this error, if so, are you able to share?
>>
>> Kind Regards,
>>
>> Nathan
>>
>> Get Outlook for iOS <https://aka.ms/o0ukef>
>> ------------------------------
>> *From:* Vibhath Ileperuma <vi...@gmail.com>
>> *Sent:* Sunday, July 31, 2022 8:49:04 AM
>> *To:* users@nifi.apache.org <us...@nifi.apache.org>
>> *Subject:* Multiplying a record filed value by a constant
>>
>> Hi All,
>>
>> I'm using Nifi's expression language with 'Update Record' processor to
>> multiply an integer value in a record path by a constant. Note that my
>> input is a csv file and I'm providing the schema through a Avro schema
>> registry. This filed I'm using is specified as a nullable filed in the
>> schema as below and this filed is null in some records.
>>
>>
>>
>>
>>
>>
>> * { "type": [ "long", "null"
>> ], "name": "Insert_Timestamp" }*
>>
>> What I want do is to multiply the filed value when it is not null and do
>> nothing when it is null.
>>
>> When I simply multiply the value as below, it throws an error when the
>> value is null.
>> *${field.value:multiply(1000000)}*
>>
>> Hence I tried to use a null check and a IfElse before multiplying as
>> below.
>> *${field.value:isNull():ifElse(${field.value} ,
>> ${field.value:multiply(1000000)})}*
>>
>> Then also I'm getting an error when the value is null.
>>
>> *UpdateRecord[id=01821014-6ce9-14b9-4604-6eea727a933b] Failed to process StandardFlowFileRecord[uuid=abd624d4-d571-49c8-9b1a-5e171b487e33,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1659246191387-1, container=default, section=1], offset=0, length=23202895],offset=0,name=abc.csv,size=23202895]; will route to failure: For input string: ""*
>>
>>
>> Could you please let me know whether there is a way to mutiply the record only if it is not null.
>>
>> Thank You.
>>
>> Best Regards,
>>
>> Vibhath Ileperuma.
>>
>