[jira] [Updated] (IGNITE-980) Investigate if we should close() the FileSystems in MapRed task implementations.

["Ivan Veselovsky (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IGNITE-980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ivan Veselovsky updated IGNITE-980:
-----------------------------------
    Description: 
Search for usages of org.apache.ignite.internal.processors.hadoop.HadoopUtils#fileSystemForMrUser and see if returned Fs should be closed.
Now the policy is straightforward: if we take the Fs with caching, we don't close it. If we take it w/o caching, we close it.
We also may add closing of all the cached instances upon Ignite node top.

  was:
FileSystem is taken as a local variable and used in the following methods.
Need to check if we should do that as try (FileSystem fs = ...) { ... }, of use the Fs without closing.

1) org.apache.ignite.internal.processors.hadoop.v2.HadoopV2JobResourceManager#cleanupStagingDirectory
2) org.apache.ignite.internal.processors.hadoop.v2.HadoopV2JobResourceManager#processFiles

3) org.apache.ignite.internal.processors.hadoop.v2.HadoopV2JobResourceManager#prepareJobEnvironment

4) org.apache.ignite.hadoop.fs.IgniteHadoopFileSystemCounterWriter#write 


> Investigate if we should close() the FileSystems in MapRed task implementations.
> --------------------------------------------------------------------------------
>
>                 Key: IGNITE-980
>                 URL: https://issues.apache.org/jira/browse/IGNITE-980
>             Project: Ignite
>          Issue Type: Task
>          Components: hadoop
>    Affects Versions: sprint-5
>            Reporter: Ivan Veselovsky
>            Assignee: Ivan Veselovsky
>            Priority: Minor
>             Fix For: sprint-6
>
>
> Search for usages of org.apache.ignite.internal.processors.hadoop.HadoopUtils#fileSystemForMrUser and see if returned Fs should be closed.
> Now the policy is straightforward: if we take the Fs with caching, we don't close it. If we take it w/o caching, we close it.
> We also may add closing of all the cached instances upon Ignite node top.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)