[jira] [Commented] (NIFI-9391) MergeRecord does not work in stateless flows

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17468721#comment-17468721 ] 

ASF subversion and git services commented on NIFI-9391:
-------------------------------------------------------

Commit 4346dd8faf692968716669117e799afadc60150c in nifi's branch refs/heads/main from Peter Turcsanyi
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4346dd8 ]

NIFI-9391: Modified MergeRecord to process FlowFiles within a loop in… (#5550)

* NIFI-9391: Modified MergeRecord to process FlowFiles within a loop in a single onTrigger

MergeRecord processed the FlowFiles in multiple onTrigger-s and it needed an extra onTrigger call
(with no incoming FFs) to realize that no more FFs are available and it is time to send the merged FF downstream.
It was not compatible with Stateless Runtime which does not trigger the flow any more if no FFs available.

Also changed "unschedule" logic in StandardProcessorTestRunner: @OnUnscheduled methods were called immediately after
the 1st FlowFile was processed. Unschedule the processor only at the end of the execution (onTrigger finished)
and only if stopOnFinish has been requested by the test case.


> MergeRecord does not work in stateless flows
> --------------------------------------------
>
>                 Key: NIFI-9391
>                 URL: https://issues.apache.org/jira/browse/NIFI-9391
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>            Reporter: Peter Turcsanyi
>            Assignee: Peter Turcsanyi
>            Priority: Major
>              Labels: stateless
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> The stateless runtime triggers a processor up until FFs are available in the upstream queue.
> MergeRecord needs an extra onTrigger() call where session.get() yields no result in order to get notified that no more FFs are available and it is time to send the merged FF downstream. This extra call will never happen in stateless environment and MergeRecord cannot send out FF in this way.
> Modify MergeRecord to run session.get() in a loop within a single onTrigger() call like MergeContent does:
> {code:java}
> while (binManager.getBinCount() <= context.getProperty(MAX_BIN_COUNT).asInteger()) {
>     if (!isScheduled()) {
>         break;
>     }
>     final ProcessSession session = sessionFactory.createSession();
>     final List<FlowFile> flowFiles = session.get(1000);
>     if (flowFiles.isEmpty()) {
>         break;
>     }
>     ... 
> {code}
> NIFI-9390 also affects MergeContent when it is not in the starting position in the flow but this jira needs to be fixed first (otherwise the other cannot pop up).



--
This message was sent by Atlassian Jira
(v8.20.1#820001)