[jira] [Resolved] (NIFI-11924) Closing HDFS file system in resource provider after using

["Peter Turcsanyi (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-11924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Peter Turcsanyi resolved NIFI-11924.
------------------------------------
    Fix Version/s: 2.0.0
                   1.24.0
       Resolution: Fixed

> Closing HDFS file system in resource provider after using
> ---------------------------------------------------------
>
>                 Key: NIFI-11924
>                 URL: https://issues.apache.org/jira/browse/NIFI-11924
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>            Reporter: Simon Bence
>            Assignee: Simon Bence
>            Priority: Critical
>             Fix For: 2.0.0, 1.24.0
>
>          Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> The {{HDFSExternalResourceProvider}} does not close the HDFS {{FileSystem}} after usage. This can lead to memory leakage. This needs to be solved. Possible solutions:
> I.: Maintain a single {{FileSystem}} resource during the lifecycle of the provider and monitor its health
> II.: Open and close a new {{FileSystem}} evert time the provider connects with the HDFS
> The first approach is considered a more error prone one as the Provider is active during the whole runtime of NiFi, thus additional logic would be needed to monitor the health of the resource and replace it in case of need. The second approach results a simpler code on the price of the performance: opening a new connection every time is subpar compared to the first option and in case of a processor which potentially works with high number of flow files within a short amount of time would be an important factor. The usage of the provider hovewer comes with a small number of calls, not resulting significant impact on the performance on NiFi nodes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)