[GitHub] [hadoop] ayushtkn commented on a change in pull request #3634: HADOOP-17997. RBF: Namespace usage of mount table with multi subclust…

[GitBox <gi...@apache.org>]

ayushtkn commented on a change in pull request #3634:
URL: https://github.com/apache/hadoop/pull/3634#discussion_r747676443



##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Quota.java
##########
@@ -50,7 +50,7 @@ public static Counts newInstance() {
    * The quota is violated if quota is set and usage > quota.
    */
   public static boolean isViolated(final long quota, final long usage) {
-    return quota >= 0 && usage > quota;
+    return quota >= 0 && usage >= quota;

Review comment:
       I think the method is correct only, It is used for LOGS in core HDFS parts, and Quota is said to be violated when the count is more than the value set, if the fileAndDirCount is 5 and quota is also 5, then quota isn't said to be violated.
   The javadoc also tells that.
   
   Regarding RBF, I think you need to change the call here to pass the delta, while verifying  :
   https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterQuotaUsage.java#L99
   
   Rather than the method which you tweaked use the below one with proper delta:
   ```
     static boolean isViolated(final long quota, final long usage,
         final long delta) {
       return quota >= 0 && delta > 0 && usage > quota - delta;
     }
   ```
   That I think should solve the problem here. Not sure how things will work in case of storage quota though....
   
   Wrote a UT in **TestRouterRPCMultipleDestinationMountTableResolver** , to check this, putting it up here as well in case anyone wants to try, might save some efforts.
   
   ```
     @Test
     public void testQuotaMultiDest() throws Exception {
       // NS-Quota as 4
       long nsQuota = 4;
       Path path = new Path("/test10");
       nnFs0.mkdirs(path);
       nnFs1.mkdirs(path);
       Map<String, String> destMap = new HashMap<>();
       destMap.put("ns0", "/test10");
       destMap.put("ns1", "/test10");
       MountTable addEntry = MountTable.newInstance("/test10", destMap);
       assertTrue(addMountTable(addEntry));
       RouterQuotaUpdateService updateService =
           routerContext.getRouter().getQuotaCacheUpdateService();
       updateService.periodicInvoke();
   
       // Set NS-Quota as 4
       RouterAdmin admin = getRouterAdmin();
       String[] argv = new String[] {"-setQuota", path.toString(), "-nsQuota",
           String.valueOf(nsQuota)};
       assertEquals(0, ToolRunner.run(admin, argv));
       updateService.periodicInvoke();
       resolver.loadCache(true);
   
       // Touch two files under ns-fed/tes10
   
       routerFs.create(new Path("/test10/file1")).close();
       routerFs.create(new Path("/test10/file2")).close();
   
       updateService.periodicInvoke();
   
       // This should have thrown exception since 2 target directories & 2 files: total 4 & Quota is 4
       routerFs.create(new Path("/test10/file3")).close();
     }
   ```
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org