Thrift gateway - status? Compared to other alternatives?

[Erik Forsberg <fo...@opera.com>]

Hi!

What's the status of the Thrift gateway for HDFS
(http://wiki.apache.org/hadoop/HDFS-APIs)? My google karma seems to be
bad because I'm not getting many hits from people using it. 

Is it stable? How's the performance? 

Assuming we're building a web interface that needs to read some files
from HDFS, and we don't want to use the Java API, would the Thrift
gateway be the best option? Or is fuse-dfs better? How about using the
http APIs for listing and fetching data, how will they compare
performance-wise?

Thanks,
\EF

-- 
Erik Forsberg <fo...@opera.com>
Developer, Opera Software - http://www.opera.com/

Re: Thrift gateway - status? Compared to other alternatives?

[Carlos Valiente <su...@gmail.com>]

On Thu, May 20, 2010 at 09:29, Erik Forsberg <fo...@opera.com> wrote:
> Assuming we're building a web interface that needs to read some files
> from HDFS, and we don't want to use the Java API, would the Thrift
> gateway be the best option? Or is fuse-dfs better?

I wrote namenode and datanode plugins to expose HDFS through Thrift
long ago. I then moved to fuse-hdfs because of performance issues
(reading data through Thrift was slower than through libhdfs, which is
what dfs-fuse does).

> How about using the
> http APIs for listing and fetching data, how will they compare
> performance-wise?

I remember having done some tests, and reads through the HTTP API were
also slower than libhdfs reads.

C