You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Konstantin Shvachko (Jira)" <ji...@apache.org> on 2020/12/13 21:12:00 UTC
[jira] [Comment Edited] (HDFS-15567) [SBN Read] HDFS should expose
msync() API to allow downstream applications call it explicitly.
[ https://issues.apache.org/jira/browse/HDFS-15567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210550#comment-17210550 ]
Konstantin Shvachko edited comment on HDFS-15567 at 12/13/20, 9:11 PM:
-----------------------------------------------------------------------
Thanks for taking a look [~vagarychen].
* Both {{FileSystem}} and {{AbstractFileSystem}} throw {{UnsupportedOperationException}} with my patch. This is a standard pattern and a way for clients to learn if the operation is supported or not in the implementation. No-op will hide the problem and for {{msync}} in particular can lead to inconsistent results further down the road, which is hard to debug as we both know.
* Logging in the tests is not "required", but it helped a lot in debugging problems that I fixed when some tests were failing. I decided to leave them in the code in case something breaks in the future. I agree we usually try to restrict change to bare minimum to avoid conflicts while backporting. In this case with code relatively recent I don't see it a blocker for backports.
* Ran tests that failed on Jenkins locally. All passed. They are long running tests, which frequently fail on Jenkins builds.
was (Author: shv):
Thanks for taking a look [~vagarychen].
* Both {{FileSystem}} and {{AbstractFileSystem}} throw {{UnsupportedOperationException}} with my patch. This is a standard pattern and a way for clients to learn if the operation is supported or not in the implementation. No-op will hide the problem and for {{mscyn}} in particular can lead to inconsistent results further down the road, which is hard to debug as we both know.
* Logging in the tests is not "required", but it helped a lot in debugging problems that I fixed when some tests were failing. I decided to leave them in the code in case something breaks in the future. I agree we usually try to restrict change to bare minimum to avoid conflicts while backporting. In this case with code relatively recent I don't see it a blocker for backports.
* Ran tests that failed on Jenkins locally. All passed. They are long running tests, which frequently fail on Jenkins builds.
> [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly.
> ----------------------------------------------------------------------------------------------
>
> Key: HDFS-15567
> URL: https://issues.apache.org/jira/browse/HDFS-15567
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: ha, hdfs-client
> Affects Versions: 2.10.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Priority: Major
> Fix For: 3.3.1, 3.4.0, 3.1.5, 2.10.2, 3.2.3
>
> Attachments: HDFS-15567.001.patch, HDFS-15567.002.patch
>
>
> Consistent reads from Standby introduced {{msync()}} API HDFS-13688, which updates client's state ID with current state of the Active NameNode to guarantee consistency of subsequent calls to an ObserverNode. Currently this API is exposed via {{DFSClient}} only, which makes it hard for applications to access {{msync()}}. One way is to use something like this:
> {code}
> if(fs instanceof DistributedFileSystem) {
> ((DistributedFileSystem)fs).getClient().msync();
> }
> {code}
> This should be exposed both for {{FileSystem}} and {{FileContext}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org