[jira] [Created] (HDFS-17473) [FGL] Make quota related operations thread-safe

["ZanderXu (Jira)" <ji...@apache.org>]

ZanderXu created HDFS-17473:
-------------------------------

             Summary: [FGL] Make quota related operations thread-safe 
                 Key: HDFS-17473
                 URL: https://issues.apache.org/jira/browse/HDFS-17473
             Project: Hadoop HDFS
          Issue Type: Sub-task
            Reporter: ZanderXu
            Assignee: ZanderXu


Concurrent operations on directory tree may cause Quota updates and verification to not be thread-safe.

For example: 
 # Supposing there is directory _/a/b_ and quota is on inode _a_ and _b_
 # There are some directories and files under {_}/a/b{_}, such as: {_}/a/b/c/d1{_}, _/a/b/d/f1.txt_
 # Supposing there is a create operation under _/a/b/c/d1_ and there is a addBlock operation on _/a/b/d/f1.txt_
 # These two operations can be handled concurrently by namenode
 # They will update the quota on inode a concurrently since these operations just hold the read lock of the inode _a_ and {_}b{_}.
 # so we should make quota-related thread safe.

 

There are two solutions to make quota-related thread safe。

Solution one: Hold the write lock of the first iNode with Quota set when resolvePath
 * Directly hold the write lock of iNode _a_ so that all operations involving subtree _/a_ can be handled safety.
 * Due to lower concurrency, maximum improvements cannot be achieved.
 * But the implementation is simple and straightforward.

Solution two: Lock all QuotaFeatures during quota verification or update
 * Still hold the read lock of iNode a and b
 * Lock all QuotaFeatures involved in this operations, when validating or updating quotas.
 * Maximum improvements can be achieved.
 * But the implementation is a little complex
 ** Add a lock for each QuotaFeature
 ** Acquire locks for all involving QuotaFeature



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org