You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Matt Burgess (JIRA)" <ji...@apache.org> on 2017/09/05 17:47:00 UTC
[jira] [Created] (NIFI-4349) AbstractRouteRecord needs to clean up
resources on error
Matt Burgess created NIFI-4349:
----------------------------------
Summary: AbstractRouteRecord needs to clean up resources on error
Key: NIFI-4349
URL: https://issues.apache.org/jira/browse/NIFI-4349
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Reporter: Matt Burgess
Reproduction environment:
CSVReader with access strategy set to "Use String Fields from Header"
CSVRecordSetWriter with Schema Write Strategy set to "Write schema.name attribute" <-- NOTE: This is an incorrect configuration, explained below
IPLookupService with local test MaxMind DB file
GenerateFlowFile -> LookupRecord -> LogAttribute
The IP addresses in the flow file are not present in the DB, but that doesn't seem to be the issue (I'm just pointing it out because it shows up in the log below). Instead, when the CSVRecordSetWriter tries to write the schema.name attribute, it can't because no schema name was ever chosen (the incoming schema is defined by the CSV header). However, rather than routing the incoming flow file to failure (which looks like the intent of the code), instead the session is rolled back and the incoming flow file returned to the queue. This is because the output flow file had been created but not removed on the following error:
2017-09-05 13:24:36,776 WARN [Timer-Driven Process Thread-3] o.a.n.c.t.ContinuallyRunProcessorTask
org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=2d59a235-234b-40f0-9d1d-edb57682e71c,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1504632264821-1, container=default, section=1], offset=285, length=0],offset=0,name=10043674590562,size=0] transfer relationship not specified
at org.apache.nifi.controller.repository.StandardProcessSession.checkpoint(StandardProcessSession.java:251)
at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:321)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:28)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1119)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2017-09-05 13:24:37,778 WARN [Timer-Driven Process Thread-3] o.a.nifi.lookup.maxmind.IPLookupService IPLookupService[id=530bf885-015e-1000-bd99-e47601e7d170] Could not resolve the IP for value '{ip=ip}'. This is usually caused by issue resolving the appropriate DNS record or providing the service with an invalid IP address:
2017-09-05 13:24:37,779 ERROR [Timer-Driven Process Thread-3] o.a.n.processors.standard.LookupRecord LookupRecord[id=53067120-015e-1000-9cb1-49ff6287d8ce] Failed to process StandardFlowFileRecord[uuid=6e3b6159-885e-449d-80ab-919cd808941c,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1504632264821-1, container=default, section=1], offset=0, length=285],offset=0,name=10043674590562,size=285]: org.apache.nifi.processor.exception.ProcessException: Could not parse incoming data
org.apache.nifi.processor.exception.ProcessException: Could not parse incoming data
at org.apache.nifi.processors.standard.AbstractRouteRecord$1.process(AbstractRouteRecord.java:151)
at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2174)
at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2144)
at org.apache.nifi.processors.standard.AbstractRouteRecord.onTrigger(AbstractRouteRecord.java:120)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1119)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.nifi.schema.access.SchemaNotFoundException: Cannot write Schema Name As Attribute because the Schema Name is not known
at org.apache.nifi.schema.access.SchemaNameAsAttribute.validateSchema(SchemaNameAsAttribute.java:53)
at org.apache.nifi.serialization.SchemaRegistryRecordSetWriter.getSchemaAccessWriter(SchemaRegistryRecordSetWriter.java:137)
at org.apache.nifi.csv.CSVRecordSetWriter.createWriter(CSVRecordSetWriter.java:73)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:89)
at com.sun.proxy.$Proxy134.createWriter(Unknown Source)
at org.apache.nifi.processors.standard.AbstractRouteRecord$1.process(AbstractRouteRecord.java:138)
... 15 common frames omitted
I believe the incoming flow file should be routed to failure and the output flow file removed from the session (and its OutputStream closed) in order to be able to commit the session successfully.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)