[jira] [Commented] (NIFI-516) MergeContent when merging 10,000s or more objects can cause too many open files

["Mark Payne (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946840#comment-14946840 ] 

Mark Payne commented on NIFI-516:
---------------------------------

Hey [~JPercivall] thanks for knocking this out! Looks good, for the most part, but one section of the patch concerns me just a bit:

{code}
+
+                // Allow processors to close the file after reading to avoid too many files open or do smart session stream management.
+                if(!allowSessionStreamManagement){
+                    currentReadClaimStream.close();
+                }
{code}

If we close that read claim stream, we need to ensure that we also then set the currentReadClaimStream to null and also null out the currentReadClaim. Otherwise, some paths through the code could end up trying to read from a stream that has already been closed.


> MergeContent when merging 10,000s or more objects can cause too many open files
> -------------------------------------------------------------------------------
>
>                 Key: NIFI-516
>                 URL: https://issues.apache.org/jira/browse/NIFI-516
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>            Reporter: Joseph Witt
>            Assignee: Joseph Percivall
>             Fix For: 0.4.0
>
>         Attachments: NIFI-516.patch, generateMerge.xml
>
>
> MergeContent really is just exposing an issue where the process session keeps file handles open longer than necessary.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)