You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Chris Nauroth (JIRA)" <ji...@apache.org> on 2017/04/13 00:58:41 UTC
[jira] [Comment Edited] (HADOOP-13726) Enforce that FileSystem
initializes only a single instance of the requested FileSystem.
[ https://issues.apache.org/jira/browse/HADOOP-13726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15966754#comment-15966754 ]
Chris Nauroth edited comment on HADOOP-13726 at 4/13/17 12:57 AM:
------------------------------------------------------------------
My understanding is that with use of the Guava {{Cache}}, we'd effectively achieve per-FS locking granularity already, not a lock over the whole {{Cache}} instance, and we'd achieve it without changing state tracking around the FS instances. Multiple threads attempting to retrieve the same FS would block waiting for the first thread to finish initialization, but threads retrieving a different FS could proceed concurrently.
If you're trying to achieve it without using the Guava {{Cache}} at all, then maybe the {{FileSystem#initialize}} call could be moved inside a {{synchronized}} block on the {{Cache.Key}} instance, and it still wouldn't need state tracking changes around the FS?
was (Author: cnauroth):
My understanding is that with use of the Guava {{Cache}}, we'd effectively achieve per-FS locking granularity already, not a lock over the whole {{Cache}} instance, and we'd achieve it without changing state tracking around the FS instances. Multiple threads attempting to retrieve the same FS would block waiting for the first thread to finish initialization, but threads retrieving a different FS could proceed concurrently.
If you're trying to achieve it without using the Guava {{Cache}} at all, then maybe the {{FileSystem#initialize}} call could be moved inside a {{syncrhonized}} block on the {{Cache.Key}} instance, and it still wouldn't need state tracking changes around the FS?
> Enforce that FileSystem initializes only a single instance of the requested FileSystem.
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-13726
> URL: https://issues.apache.org/jira/browse/HADOOP-13726
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Reporter: Chris Nauroth
> Assignee: Manjunath Anand
>
> The {{FileSystem}} cache is intended to guarantee reuse of instances by multiple call sites or multiple threads. The current implementation does provide this guarantee, but there is a brief race condition window during which multiple threads could perform redundant initialization. If the file system implementation has expensive initialization logic, then this is wasteful. This issue proposes to eliminate that race condition and guarantee initialization of only a single instance.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org