[jira] [Commented] (HADOOP-12910) Add new FileSystem API to support asynchronous method calls

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-12910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310247#comment-15310247 ] 

Steve Loughran commented on HADOOP-12910:
-----------------------------------------

I'm going to highilght that vast quantities of code people rely on is tagged as "Unstable"; 

having things marked as such is not a green light to cut things on a whim. It's always good to check across the downstream projects who is actually using a method or interface before breaking things.

I've been thinking we need an extra unstable tag, {{@Experimental}}, which would mean "this entire feature could be removed without warning". This async API would fit that category

bq. The down streams are intelligent people. They can decide whether they want to use the unstable API. 

It's always insightful to work downstream, especially downstream code you are trying to get to compile and work against multiple versions. HDFS changes are things I fear, though generally it's the new packaging changes rather than any interface or behaviour; that's more at the YARN level. And we can't say "never trust unstable" or "never use private/limited private" as its impossible to do things. We end up picking things up not noticing their stability guarantees (i.e. we cut and paste from working code and test cases), or pull it in without scanning the entire tree of dependent classes.

Here are some examples of things downstream apps depend on

* UGI {{@LimitedPrivate, @Evolving}}
* the YARN API records: {{ApplicationAttemptReport, ContainerReport, ContainerExitStatus}} @Unstable

These are things we use every day, and we don't make a conscious decision in the expectation that it will suddenly vanish, more that "we hope they don't break". 

if the stability of a new API is less, then I think an @Experimental tag would be good. Ship it, learn from the experience, be prepared to rewrite. And the tag would make clear that this stuff is really, really unstable. Then follow through by removing the @Experimental tag once it's no longer considered an experiment: this stops the tag becoming as devalued as the rest are.




(holding off on any opinion about the API, just highlighting that there are issues with our tags, and we cannot treat private/limited private and unstable as hardcoded freedom to play, not without discovering how things get used. This is why [~stack]'s comments are so welcome.


> Add new FileSystem API to support asynchronous method calls
> -----------------------------------------------------------
>
>                 Key: HADOOP-12910
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12910
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Xiaobing Zhou
>         Attachments: HADOOP-12910-HDFS-9924.000.patch, HADOOP-12910-HDFS-9924.001.patch, HADOOP-12910-HDFS-9924.002.patch
>
>
> Add a new API, namely FutureFileSystem (or AsynchronousFileSystem, if it is a better name).  All the APIs in FutureFileSystem are the same as FileSystem except that the return type is wrapped by Future, e.g.
> {code}
>   //FileSystem
>   public boolean rename(Path src, Path dst) throws IOException;
>   //FutureFileSystem
>   public Future<Boolean> rename(Path src, Path dst) throws IOException;
> {code}
> Note that FutureFileSystem does not extend FileSystem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org