You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Eli Collins <el...@cloudera.com> on 2011/01/02 23:28:50 UTC

Re: *FileSystem and *Fs APIs

Most of the Hadoop code base (MapReduce, FsShell etc) has not yet
switched to the new API yet.

On Fri, Dec 31, 2010 at 9:16 AM, Zhenhua Guo <je...@gmail.com> wrote:
> Eli,
> Thanks.
> So the new API is being used by default in current version of Hadoop,
> e.g. 0.21.0 and up?
>
> Gerald
>
> On Fri, Dec 31, 2010 at 12:06 AM, Eli Collins <el...@cloudera.com> wrote:
>> Hey Gerald,
>>
>> Yes, there's a new API. FileSystem is replaced by two new classes:
>> FileContext (the user-facing API) and AbstractFileSystem (the fs
>> implementation API). So the AbstractFileSystem derived classes replace
>> the FileSystem derived classes.
>>
>> Thanks,
>> Eli
>>
>> On Thu, Dec 30, 2010 at 8:16 PM, Zhenhua Guo <je...@gmail.com> wrote:
>>> I noticed that in HDFS there are two sets of API classes - *FileSystem
>>> and *FS, e.g. FtpFileSystem vs. FtpFs, ChecksumFileSystem vs.
>>> ChecksumFS. I wonder what is the difference. One is replacement of the
>>> other?
>>>
>>> Thanks
>>> Gerald
>>>
>>
>

Re: *FileSystem and *Fs APIs

Posted by Eli Collins <el...@cloudera.com>.
On Sun, Jan 2, 2011 at 2:32 PM, Noah Watkins <ja...@cs.ucsc.edu> wrote:
>> Most of the Hadoop code base (MapReduce, FsShell etc) has not yet
>> switched to the new API yet.
>
> I have a FS implementation that is nearly complete, and designed against the
> 0.21 code base, using the old FileSystem interface. Do you have any
> recommendations on how to move to the new API in trunk?
>

Porting from FileSystem to FileContext/AbstractFileSystem is fairly
straight forward as the APIs are similar (though now split across two
classes). If you're just implementing a new file system you just
switch from using FileSystem to AbstractFileSystem (eg checkout
ChecksumFileSystem vs ChecksumFs).

Thanks,
Eli

Re: *FileSystem and *Fs APIs

Posted by Noah Watkins <ja...@cs.ucsc.edu>.
> Most of the Hadoop code base (MapReduce, FsShell etc) has not yet
> switched to the new API yet.

I have a FS implementation that is nearly complete, and designed against the
0.21 code base, using the old FileSystem interface. Do you have any
recommendations on how to move to the new API in trunk?

Thanks,
Noah