You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Aaron Fabbri (JIRA)" <ji...@apache.org> on 2018/04/20 23:04:00 UTC
[jira] [Commented] (HADOOP-13649) s3guard: implement time-based
(TTL) expiry for LocalMetadataStore
[ https://issues.apache.org/jira/browse/HADOOP-13649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446485#comment-16446485 ]
Aaron Fabbri commented on HADOOP-13649:
---------------------------------------
Assigning to [~gabor.bota]. Feel free to change the datastructures used. The "two hashtables" was a quick and dirty design that may not be the best if we add TTL-based expiry. I would look into [~stevel@apache.org]'s suggestion to use Guava cache as well. Not sure if it has the needed functionality to support prune and all the other operations that go with it (note that prune() sometimes needs to clear isAuthorititative bit on a directory if a file in that directory is pruned).
> s3guard: implement time-based (TTL) expiry for LocalMetadataStore
> -----------------------------------------------------------------
>
> Key: HADOOP-13649
> URL: https://issues.apache.org/jira/browse/HADOOP-13649
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.0.0-beta1
> Reporter: Aaron Fabbri
> Assignee: Aaron Fabbri
> Priority: Minor
>
> LocalMetadataStore is primarily a reference implementation for testing. It may be useful in narrow circumstances where the workload can tolerate short-term lack of inter-node consistency: Being in-memory, one JVM/node's LocalMetadataStore will not see another node's changes to the underlying filesystem.
> To put a bound on the time during which this inconsistency may occur, we should implement time-based (a.k.a. Time To Live / TTL) expiration for LocalMetadataStore
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org