You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Ayush Saxena (Jira)" <ji...@apache.org> on 2020/11/05 11:01:00 UTC
[jira] [Updated] (HDFS-15669) RBF: Incorrect GetQuota caused by
different implementation of HashSet
[ https://issues.apache.org/jira/browse/HDFS-15669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ayush Saxena updated HDFS-15669:
--------------------------------
Summary: RBF: Incorrect GetQuota caused by different implementation of HashSet (was: [HDFS] [RBF] Incorrect GetQuota caused by different implementation of HashSet )
> RBF: Incorrect GetQuota caused by different implementation of HashSet
> ----------------------------------------------------------------------
>
> Key: HDFS-15669
> URL: https://issues.apache.org/jira/browse/HDFS-15669
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Janus Chow
> Priority: Major
>
> In the method of Quota#getQuotaUsage, the result could be different on different versions of Java.
> The trace route is as follows:
> {code:java}
> Quota#getQuotaUsage
> - Quota#getValidQuotaLocations
> - Quota#getQuotaRemoteLocations
> - RouterQuotaManager#getPaths{code}
> In RouterQuotaManager#getPaths, the path is stored in a _HashSet_. And in Quota#getValidQuotaLocations, the values in HashSet would be used to check if the paths are parent-child relation. So paths in HashSet could affect the result of getValidQuotaLocations as follows:
> {code:java}
> // Situation.1 Locations in HashSet
> [ns0->/testdir7, ns0->/testdir7/subdir, ns1->/testdir8]
> // Situation.1 getQuota results
> {ns0->/testdir7= 10 6 100 100 , ns1->/testdir8= 10 8 100 100}
> // Situation.2 Locations in HashSet
> [ns0->/testdir7/subdir, ns1->/testdir8, ns0->/testdir7]
> // Situation.2 getQuota results
> {ns0->/testdir7= 10 8 100 100 , ns0->/testdir7/subdir= 10 6 100 100 , ns1->/testdir8= 10 8 100 100 }{code}
> Situation.1 and Situation.2 happen when the underlying implementation of HashSet is different, that is Situation.2 happens using Java-7, and Situation.2 happens after Java-8.
> This problem can be solved when we do sorting on the results of Quota#_getQuotaRemoteLocations_, but I'm not sure if we should do it.
> {code:java}
> /**
> * Get all quota remote locations across subclusters under given
> * federation path.
> * @param path Federation path.
> * @return List of quota remote locations.
> * @throws IOException
> */
> private List<RemoteLocation> getQuotaRemoteLocations(String path)
> throws IOException {
> List<RemoteLocation> locations = new LinkedList<>();
> RouterQuotaManager manager = this.router.getQuotaManager();
> if (manager != null) {
> Set<String> childrenPaths = manager.getPaths(path);
> for (String childPath : childrenPaths) {
> locations.addAll(rpcServer.getLocationsForPath(childPath, true, false));
> }
> }
> ++ Collections.sort(locations);
> return locations;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org