[jira] [Commented] (YARN-3874) Combine FS Reader and Writer Implementations

["Varun Saxena (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-3874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14635115#comment-14635115 ] 

Varun Saxena commented on YARN-3874:
------------------------------------

[~sjlee0], [~zjshen], kindly review.

Following has been done :
# Both FS reader and writer implementations have been made consistent with each other. Classes have not yet been combined as no final decision was taken on it in YARN-3051. We can decide if we will combine the classes or not.
# Moved some of the code common to reader and writer impls into {{TimelineStorageUtils}}
# Writer impl will now write app flow mapping file.
# As [~zjshen] said in YARN-3051, used {{FileSystem}} class so that it can be used for HDFS as well.
# Entity files would now have created time in the file name. This will be used to limit the entities returned and do filtering based on created time.
# Modified times as of now, wont be part of file name as we will have to consistently change filenames if we do that. If required, that can be included as well. We can use rename operation. Thoughts ?
# Writer would choose the file based on entity ID if created time is not given in request. If created time is 0 or negative and file for an entity id is being written for the first time, an error will be sent back.
# App flow mapping will be cached in reader and writer on start. And entries will be added in reader whenever mapping has to be queried from file


> Combine FS Reader and Writer Implementations
> --------------------------------------------
>
>                 Key: YARN-3874
>                 URL: https://issues.apache.org/jira/browse/YARN-3874
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-3874-YARN-2928.01.patch
>
>
> Combine FS Reader and Writer Implementations and make them consistent with each other.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)