You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hbase.apache.org by "mazhenlin (JIRA)" <ji...@apache.org> on 2018/10/18 18:10:00 UTC

[jira] [Created] (HBASE-21342) FileSystem in use was closed by others in secure bulkLoad

mazhenlin created HBASE-21342:
---------------------------------

             Summary: FileSystem in use was closed by others  in secure bulkLoad
                 Key: HBASE-21342
                 URL: https://issues.apache.org/jira/browse/HBASE-21342
             Project: HBase
          Issue Type: Bug
    Affects Versions: 1.2.7, 2.0.1, 1.4.4, 2.1.0, 3.0.0, 1.5.0, 1.3.3
            Reporter: mazhenlin
         Attachments: race.patch

As mentioned in [#HBASE-15291], there is a race condition.   If Two secure bulkload calls  from the same UGI into two different regions and one region finishes earlier, it will close the bulk load fs, and the other region will fail.

 

Another case would be more serious. The FileSystem.close() function need two synchronized variables : CACHE and deleteOnExit. If one region calls FileSystem.closeAllForUGI ( in SecureBulkLoadManager.cleanupBulkLoad) while another region is trying to close srcFS ( in  SecureBulkLoadListener.closeSrcFs)   , can cause deadlock here.

 

I have wrote a UT for this and fixed it using reference counter.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)