[jira] [Commented] (OOZIE-1683) UserGroupInformationService should close any filesystems opened by it

["Hadoop QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-1683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13886209#comment-13886209 ] 

Hadoop QA commented on OOZIE-1683:
----------------------------------

Testing JIRA OOZIE-1683

Cleaning local git workspace

----------------------------

{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.    {color:green}+1{color} the patch does not introduce any @author tags
.    {color:green}+1{color} the patch does not introduce any tabs
.    {color:green}+1{color} the patch does not introduce any trailing spaces
.    {color:green}+1{color} the patch does not introduce any line longer than 132
.    {color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.    {color:green}+1{color} the patch does not seem to introduce new RAT warnings
{color:green}+1 JAVADOC{color}
.    {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings
{color:green}+1 COMPILE{color}
.    {color:green}+1{color} HEAD compiles
.    {color:green}+1{color} patch compiles
.    {color:green}+1{color} the patch does not seem to introduce new javac warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.    {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
.    {color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.    Tests run: 1388
.    Tests failed: 0
.    Tests errors: 1

.    The patch failed the following testcases:

.      

{color:green}+1 DISTRO{color}
.    {color:green}+1{color} distro tarball builds with the patch 

----------------------------
{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/1038/

> UserGroupInformationService should close any filesystems opened by it
> ---------------------------------------------------------------------
>
>                 Key: OOZIE-1683
>                 URL: https://issues.apache.org/jira/browse/OOZIE-1683
>             Project: Oozie
>          Issue Type: Bug
>    Affects Versions: trunk
>            Reporter: Robert Kanter
>            Assignee: Robert Kanter
>         Attachments: OOZIE-1683.patch
>
>
> When you create a FileSystem object in HDFS, it caches it.  However, when using a UGI to get the FileSystem, it caches it to that object, not its “value”.  In other words, if you create two UGIs for foo to impersonate bar, then they will each create an entry in the cache.  In Oozie, we created a UserGroupInformationService class that basically sits on top of the HDFS cache and caches the UGI objects for impersonating a user so we can reuse the same FileSystem object each time.  However, when running unit tests, we typically destroy and recreate the services for each test (sometimes even more) so Oozie’s cache was being lost and we were effectively creating a new FileSystem object all the time without closing it. 
> Looking at a heap dump after running many of the tests, we saw that there were 800+ FileSystem objects open for the same FileSystem.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)