How do I read a file to a buffer from HDFS?

[Frank LIN <wo...@gmail.com>]

Hi all,

For some reasons, I need to read a small fine into memory from HDFS. I tried:

    final Path file = new Path(...) // file name on HDFS
    byte[] buffer = new byte[100000];
    try {
        FileSystem fs = FileSystem.get(conf);
        FSDataInputStream fileIn = fs.open(file);
        fileIn.readFully(buffer);
    } catch (Exception e) {
        System.err.println(e.toString());
    }

but the "fileIn.readFully(buffer)" always throws a EOF exception.

Any suggestions on that? Thank you.


-- 
Frank Lin
Undergraduate Student
Dept. Computer Science and Technology,
Tsinghua University, Beijing, P.R.China

Re: How do I read a file to a buffer from HDFS?

[Raghu Angadi <ra...@yahoo-inc.com>]

readFully(arr) implies that you are expecting read nothing less than 
arr.length bytes. If the file does not have that many bytes, you will 
get an EOF exception. 
http://java.sun.com/j2se/1.5.0/docs/api/java/io/DataInputStream.html#readFully(byte[])

Raghu.

Frank LIN wrote:
> Hi all,
> 
> For some reasons, I need to read a small fine into memory from HDFS. I tried:
> 
>     final Path file = new Path(...) // file name on HDFS
>     byte[] buffer = new byte[100000];
>     try {
>         FileSystem fs = FileSystem.get(conf);
>         FSDataInputStream fileIn = fs.open(file);
>         fileIn.readFully(buffer);
>     } catch (Exception e) {
>         System.err.println(e.toString());
>     }
> 
> but the "fileIn.readFully(buffer)" always throws a EOF exception.
> 
> Any suggestions on that? Thank you.
> 
>