You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2018/05/02 11:21:00 UTC
[jira] [Commented] (HADOOP-15229) Add FileSystem builder-based open
API to match create()
[ https://issues.apache.org/jira/browse/HADOOP-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460889#comment-16460889 ]
Steve Loughran commented on HADOOP-15229:
-----------------------------------------
I see there's a fair few people watching this.
* I'm not working on this as I have too many other commitments
* HADOOP-14365 is a predecessor; due diligence and verification that the builder pattern works. Again, no active dev.
* As is HADOOP-13327, which helps complete the FS spec before we add more stuff. That does have my name next to it, but its a "in my free time" piece of work. I'll try and look it at some point this month, after which reviews will be welcome
Which means, if you are hoping to see this in a forthcoming release, you get control the release schedule through implementing its dependencies and then this, with docs and tests. Thx
> Add FileSystem builder-based open API to match create()
> -------------------------------------------------------
>
> Key: HADOOP-15229
> URL: https://issues.apache.org/jira/browse/HADOOP-15229
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs, fs/azure, fs/s3
> Affects Versions: 3.0.0
> Reporter: Steve Loughran
> Priority: Major
>
> Replicate HDFS-1170 and HADOOP-14365 with an API to open files.
> A key requirement of this is not HDFS, it's to put in the fadvise policy for working with object stores, where getting the decision to do a full GET and TCP abort on seek vs smaller GETs is fundamentally different: the wrong option can cost you minutes. S3A and Azure both have adaptive policies now (first backward seek), but they still don't do it that well.
> Columnar formats (ORC, Parquet) should be able to say "fs.input.fadvise" "random" as an option when they open files; I can imagine other options too.
> The Builder model of [~eddyxu] is the one to mimic, method for method. Ideally with as much code reuse as possible
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org