You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Wei-Chiu Chuang (Jira)" <ji...@apache.org> on 2019/10/01 00:01:31 UTC
[jira] [Commented] (HADOOP-16461) Regression: FileSystem cache lock
parses XML within the lock
[ https://issues.apache.org/jira/browse/HADOOP-16461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16941416#comment-16941416 ]
Wei-Chiu Chuang commented on HADOOP-16461:
------------------------------------------
Commit applies to branch-3.2 and branch-3.1.
I'll cherry pick the commit into these two branches.
> Regression: FileSystem cache lock parses XML within the lock
> ------------------------------------------------------------
>
> Key: HADOOP-16461
> URL: https://issues.apache.org/jira/browse/HADOOP-16461
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 3.3.0
> Reporter: Gopal Vijayaraghavan
> Assignee: Gopal Vijayaraghavan
> Priority: Major
> Fix For: 3.3.0
>
>
> https://github.com/apache/hadoop/blob/2546e6ece240924af2188bb39b3954a4896e4a4f/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java#L3388
> {code}
> fs = createFileSystem(uri, conf);
> synchronized (this) { // refetch the lock again
> FileSystem oldfs = map.get(key);
> if (oldfs != null) { // a file system is created while lock is releasing
> fs.close(); // close the new file system
> return oldfs; // return the old file system
> }
> // now insert the new file system into the map
> if (map.isEmpty()
> && !ShutdownHookManager.get().isShutdownInProgress()) {
> ShutdownHookManager.get().addShutdownHook(clientFinalizer, SHUTDOWN_HOOK_PRIORITY);
> }
> fs.key = key;
> map.put(key, fs);
> if (conf.getBoolean(
> FS_AUTOMATIC_CLOSE_KEY, FS_AUTOMATIC_CLOSE_DEFAULT)) {
> toAutoClose.add(key);
> }
> return fs;
> }
> {code}
> The lock now has a ShutdownHook creation, which ends up doing
> https://github.com/apache/hadoop/blob/2546e6ece240924af2188bb39b3954a4896e4a4f/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/ShutdownHookManager.java#L205
> {code}
> HookEntry(Runnable hook, int priority) {
> this(hook, priority,
> getShutdownTimeout(new Configuration()),
> TIME_UNIT_DEFAULT);
> }
> {code}
> which ends up doing a "new Configuration()" within the locked section.
> This indirectly hurts the cache hit scenarios as well, since if the lock on this is held, then the other section cannot be entered either.
> https://github.com/apache/tez/blob/master/tez-runtime-library/src/main/java/org/apache/tez/runtime/library/common/sort/impl/TezSpillRecord.java#L65
> {code}
> I/O Setup 0 State: BLOCKED CPU usage on sample: 6ms
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(URI, Configuration, FileSystem$Cache$Key) FileSystem.java:3345
> org.apache.hadoop.fs.FileSystem$Cache.get(URI, Configuration) FileSystem.java:3320
> org.apache.hadoop.fs.FileSystem.get(URI, Configuration) FileSystem.java:479
> org.apache.hadoop.fs.FileSystem.getLocal(Configuration) FileSystem.java:435
> {code}
> slowing down the RawLocalFileSystem when there are other threads creating HDFS FileSystem objects at the same time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org