You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2019/02/07 04:37:00 UTC
[jira] [Commented] (HBASE-21342) FileSystem in use may get closed
by other bulk load call in secure bulkLoad
[ https://issues.apache.org/jira/browse/HBASE-21342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16762356#comment-16762356 ]
Hudson commented on HBASE-21342:
--------------------------------
SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1202 (See [https://builds.apache.org/job/HBase-1.2-IT/1202/])
HBASE-21374 Backport HBASE-21342 to branch-1 (busbey: [https://github.com/apache/hbase/commit/f7b51ccdf7414bc1deada02e3add556dc76e97be])
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestSecureBulkLoadEndpoint.java
> FileSystem in use may get closed by other bulk load call in secure bulkLoad
> ----------------------------------------------------------------------------
>
> Key: HBASE-21342
> URL: https://issues.apache.org/jira/browse/HBASE-21342
> Project: HBase
> Issue Type: Bug
> Affects Versions: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.4.4, 2.0.1, 1.2.7
> Reporter: mazhenlin
> Assignee: mazhenlin
> Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: 21342.v1.txt, HBASE-21342.002.patch, HBASE-21342.003.patch, HBASE-21342.004.patch, HBASE-21342.005.patch, HBASE-21342.006.patch, HBASE-21342.007.patch, race.patch
>
>
> As mentioned in [HBASE-15291|#HBASE-15291], there is a race condition. If Two secure bulkload calls from the same UGI into two different regions and one region finishes earlier, it will close the bulk load fs, and the other region will fail.
>
> Another case would be more serious. The FileSystem.close() function needs two synchronized variables : CACHE and deleteOnExit. If one region calls FileSystem.closeAllForUGI ( in SecureBulkLoadManager.cleanupBulkLoad) while another region is trying to close srcFS ( in SecureBulkLoadListener.closeSrcFs) , can cause deadlock here.
>
> I have wrote a UT for this and fixed it using reference counter.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)