You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Koji Kawamura (JIRA)" <ji...@apache.org> on 2017/09/20 04:31:00 UTC
[jira] [Comment Edited] (NIFI-4395) GenerateTableFetch can't fetch
column type by state after instance reboot
[ https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16172728#comment-16172728 ]
Koji Kawamura edited comment on NIFI-4395 at 9/20/17 4:30 AM:
--------------------------------------------------------------
[~deonashh] Thanks for reporting this. I was able to reproduce the same exception with MySQL. Full stack-trace as below:
{code}
2017-09-20 13:24:17,218 WARN [Timer-Driven Process Thread-1] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding GenerateTableFetch[id=9d82e299-015e-1000-ba64-35cceeb836e5] due to uncaught Exception: java.lang.IllegalArgumentException: No column type found for: ID
2017-09-20 13:24:17,218 WARN [Timer-Driven Process Thread-1] o.a.n.c.t.ContinuallyRunProcessorTask
java.lang.IllegalArgumentException: No column type found for: ID
at org.apache.nifi.processors.standard.GenerateTableFetch.getColumnType(GenerateTableFetch.java:414)
at org.apache.nifi.processors.standard.GenerateTableFetch.lambda$onTrigger$0(GenerateTableFetch.java:246)
at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:557)
at org.apache.nifi.processors.standard.GenerateTableFetch.onTrigger(GenerateTableFetch.java:240)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1119)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
If you are able to submit a PR, that'd be really appreciated! Since I have already reproduced the issue, the review and confirm process should be quick.
was (Author: ijokarumawak):
[~deonashh]] Thanks for reporting this. I was able to reproduce the same exception with MySQL. Full stack-trace as below:
{code}
2017-09-20 13:24:17,218 WARN [Timer-Driven Process Thread-1] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding GenerateTableFetch[id=9d82e299-015e-1000-ba64-35cceeb836e5] due to uncaught Exception: java.lang.IllegalArgumentException: No column type found for: ID
2017-09-20 13:24:17,218 WARN [Timer-Driven Process Thread-1] o.a.n.c.t.ContinuallyRunProcessorTask
java.lang.IllegalArgumentException: No column type found for: ID
at org.apache.nifi.processors.standard.GenerateTableFetch.getColumnType(GenerateTableFetch.java:414)
at org.apache.nifi.processors.standard.GenerateTableFetch.lambda$onTrigger$0(GenerateTableFetch.java:246)
at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:557)
at org.apache.nifi.processors.standard.GenerateTableFetch.onTrigger(GenerateTableFetch.java:240)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1119)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
If you are able to submit a PR, that'd be really appreciated! Since I have already reproduced the issue, the review and confirm process should be quick.
> GenerateTableFetch can't fetch column type by state after instance reboot
> -------------------------------------------------------------------------
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.3.0
> Reporter: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ---
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " + colName);
> }
> ---
> When this situation happened. The FlowFile will also be grab and can't release or observed.
> Processor can't existing grab column type from columnTypeMap through instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)