Processing large file from HDFS

[Jestin Joy <je...@gmail.com>]

Unlike many users here, I started using Hadoop with Hama. I have a large
file for graph processing (Close to 1GB). What should be the best way to
read that file from HDFS? Read the whole data to RAM? The file is arranged
as

1 2
1 4
1 5
....

Where first column denotes a vertex and second column denotes adjacent
vertex to that vertex.  What is the idea of processing a large file from
HDFS?

Re: Processing large file from HDFS

["Edward J. Yoon" <ed...@apache.org>]

Please see https://hama.apache.org/hama_graph_tutorial.html and Vertex
Reader and Writer.

On Mon, Jun 16, 2014 at 2:36 PM, Jestin Joy <je...@gmail.com> wrote:
> Unlike many users here, I started using Hadoop with Hama. I have a large
> file for graph processing (Close to 1GB). What should be the best way to
> read that file from HDFS? Read the whole data to RAM? The file is arranged
> as
>
> 1 2
> 1 4
> 1 5
> ....
>
> Where first column denotes a vertex and second column denotes adjacent
> vertex to that vertex.  What is the idea of processing a large file from
> HDFS?



-- 
Best Regards, Edward J. Yoon
CEO at DataSayer Co., Ltd.