You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "zhengchenyu (Jira)" <ji...@apache.org> on 2022/08/01 11:25:00 UTC
[jira] [Comment Edited] (HDFS-13522) RBF: Support observer node from Router-Based Federation
[ https://issues.apache.org/jira/browse/HDFS-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573704#comment-17573704 ]
zhengchenyu edited comment on HDFS-13522 at 8/1/22 11:24 AM:
-------------------------------------------------------------
[~xuzq_zander]
Hi, the use case about design A is very rare indeed. But Design A also have advantage.
(1) More flexible
Client could set their msync period time by itself.
Example: In our cluster, one name service, some special user detect hdfs file is created periodically, may need high time precision, means more frequent msync.(Though I am oppose to this way).
(2) Save msync
I think there is no need to call msync periodically for most HIVE, MR application. Design A will save more msync than Design B。
I agree [~xuzq_zander] 's suggestion that focus on Design B first, add Design A as a bonus item. It is no easy to review both Design A and Design B.
Could we only complete Design B in this issue?
was (Author: zhengchenyu):
[~xuzq_zander]
Hi, the use case about design A is very rare indeed. But Design A also have advantage.
(1) More flexible
Client could set their msync period time by itself.
Example: In our cluster, one name service, some special user detect hdfs file is created periodically, may need high time precision, means more frequent msync.(Though I am oppose to this way).
(2) Save msync
I think there is no need to call msync periodically for most HIVE, MR application. Design A will save more msync than Design B。
I agree your suggestion that focus on Design B first, add Design A as a bonus item. It is no easy to review both Design A and Design B.
> RBF: Support observer node from Router-Based Federation
> -------------------------------------------------------
>
> Key: HDFS-13522
> URL: https://issues.apache.org/jira/browse/HDFS-13522
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: federation, namenode
> Reporter: Erik Krogen
> Assignee: Simbarashe Dzinamarira
> Priority: Major
> Labels: pull-request-available
> Attachments: HDFS-13522.001.patch, HDFS-13522.002.patch, HDFS-13522_WIP.patch, HDFS-13522_proposal_zhengchenyu_v1.pdf, RBF_ Observer support.pdf, Router+Observer RPC clogging.png, ShortTerm-Routers+Observer.png, observer_reads_in_rbf_proposal_simbadzina_v1.pdf, observer_reads_in_rbf_proposal_simbadzina_v2.pdf
>
> Time Spent: 20h 50m
> Remaining Estimate: 0h
>
> Changes will need to occur to the router to support the new observer node.
> One such change will be to make the router understand the observer state, e.g. {{FederationNamenodeServiceState}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org