Re: Regarding HDFS-15567. HDFS should expose msync() API to allow downstream applications call it explicitly

[Xiaoqiao He <xq...@gmail.com>]

Hi Konstantin and Steve,

HDFS-15751 has committed to related branches including 3.2.2. I will
prepare 3.2.2-RC shortly. Please let me know if any other issues for doc or
deployment do you meet. Thanks.

- He Xiaoqiao

On Wed, Dec 30, 2020 at 6:40 PM Xiaoqiao He <xq...@gmail.com> wrote:

> Hi Steve and Chao,
>
> Konstatin has updated HDFS-15751 and attached patch for documentation
> msync API, Would you mind to TAL. I want to involve this update to 3.2.2
> release, but for now it seems there are not enough committers to review.
> Please check when you have bandwidth. Thanks all.
>
> - He Xiaoqiao
>
> On Fri, Dec 25, 2020 at 5:40 AM Konstantin Shvachko <sh...@gmail.com>
> wrote:
>
>> Hi Steve,
>>
>> I created HDFS-15751 <https://issues.apache.org/jira/browse/HDFS-15751>
>> for
>> documenting msync API.
>> Would appreciate your suggestions.
>>
>> Stay safe,
>> --Konstantin
>>
>> On Mon, Dec 21, 2020 at 5:19 AM Steve Loughran <st...@cloudera.com>
>> wrote:
>>
>> >
>> >
>> > On Fri, 18 Dec 2020 at 23:29, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> Hey Steve,
>> >>
>> >> Thanks for the references. I was reading but still need to understand
>> how
>> >> exactly this applies to msync.
>> >>
>> >
>> > mainly: pull it up and it becomes part of the broader API, so needs to
>> be
>> > specified in a way which can be understood by users and for
>> implementors of
>> > others stores: to give their own stores the same semantics.
>> >
>> > What does the HDFS one do?
>> >
>> >
>> >
>> >> Will come up with a plan and post it on a new jira.
>> >> Will make sure to create it under HADOOP and ping Hadoop Common list
>> for
>> >> visibility.
>> >>
>> >>
>> > thanks
>> >
>> >
>> >> You are right about ViewFS. The impl should make sure it calls msync()
>> on
>> >> all mount points that enabled observer reads.
>> >>
>> >>
>> > That's the kind of issue this process aims to resolve. Another is to
>> > identify where we have HDFS-layer "quirks" and at least document them
>> (e.g.
>> > how hdfs streams are thread safe, rename isn't Posix, ...) and list
>> what we
>> > know breaks if you don't re-implement
>> >
>>
>