Re: loading vertices into RAM

["Edward J. Yoon" <ed...@apache.org>]

Today I tried to use off-heap vertices info but it throws Class not
found exceptions. :/ Which jar need to be included to use it?

On Wed, Jan 15, 2014 at 3:59 PM, Tommaso Teofili
<to...@gmail.com> wrote:
> and OffHeapVerticesInfo for loading vertices off heap, which is available
> with 0.6.3 as well if I recall correctly.
> Tommaso
>
>
> 2014/1/15 Edward J. Yoon <ed...@apache.org>
>
>> There are few implementations.
>>
>>  - ListVerticesinfo: loads vertices into array list.
>>  - MapVerticesinfo: loads vertices into tree map.
>>  - DiskVerticesInfo: loads vertices into a local file.
>>
>> You can choose one of them by setting the "hama.graph.vertices.info"
>> in job configuration.
>>
>>   > conf.setClass("hama.graph.vertices.info",
>> "org.apache.hama.graph.ListVerticesInfo".
>>
>> With the latest 0.6.3 version, you can use only ListVerticesInfo.
>> Please use the TRUNK.
>>
>>
>> On Tue, Jan 14, 2014 at 11:18 PM, Ammar Sahib <am...@yahoo.com>
>> wrote:
>> > Hi
>> >
>> > According to the BSP model, the data is processed in the RAM and that is
>> the reason why Pregel model is faster than the MapReduce (MapReduce
>> writedown to disk). Can someone explains to me how to be sure that all the
>> graph vertices are actually been loaded in RAM?
>> >
>> >
>> > How would HAMA behave if the vertices values are so big such that the
>> available RAM memory is not enough to contains all of the vertices?
>> >
>> > Regards
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Re: loading vertices into RAM

[Tommaso Teofili <to...@gmail.com>]

of course you would also need to add DirectMemory [1][2] (core and kryo
modules) to the classpath (e.g. if you're triggering jobs by scripts).

Tommaso

[1] :
http://repo1.maven.org/maven2/org/apache/directmemory/directmemory-cache/0.2/directmemory-cache-0.2.jar
[2] :
http://repo1.maven.org/maven2/org/apache/directmemory/directmemory-kryo/0.2/directmemory-kryo-0.2.jar


2014/1/16 Tommaso Teofili <to...@gmail.com>

> it's in hama-graph 0.6.3 [1].
>
> Tommaso
>
>
> [1] :
> http://grepcode.com/file/repo1.maven.org$maven2@org.apache.hama$hama-graph@0.6.3@org$apache$hama$graph$OffHeapVerticesInfo.java
>
>
> 2014/1/15 Edward J. Yoon <ed...@apache.org>
>
>> Today I tried to use off-heap vertices info but it throws Class not
>> found exceptions. :/ Which jar need to be included to use it?
>>
>> On Wed, Jan 15, 2014 at 3:59 PM, Tommaso Teofili
>> <to...@gmail.com> wrote:
>> > and OffHeapVerticesInfo for loading vertices off heap, which is
>> available
>> > with 0.6.3 as well if I recall correctly.
>> > Tommaso
>> >
>> >
>> > 2014/1/15 Edward J. Yoon <ed...@apache.org>
>> >
>> >> There are few implementations.
>> >>
>> >>  - ListVerticesinfo: loads vertices into array list.
>> >>  - MapVerticesinfo: loads vertices into tree map.
>> >>  - DiskVerticesInfo: loads vertices into a local file.
>> >>
>> >> You can choose one of them by setting the "hama.graph.vertices.info"
>> >> in job configuration.
>> >>
>> >>   > conf.setClass("hama.graph.vertices.info",
>> >> "org.apache.hama.graph.ListVerticesInfo".
>> >>
>> >> With the latest 0.6.3 version, you can use only ListVerticesInfo.
>> >> Please use the TRUNK.
>> >>
>> >>
>> >> On Tue, Jan 14, 2014 at 11:18 PM, Ammar Sahib <am...@yahoo.com>
>> >> wrote:
>> >> > Hi
>> >> >
>> >> > According to the BSP model, the data is processed in the RAM and
>> that is
>> >> the reason why Pregel model is faster than the MapReduce (MapReduce
>> >> writedown to disk). Can someone explains to me how to be sure that all
>> the
>> >> graph vertices are actually been loaded in RAM?
>> >> >
>> >> >
>> >> > How would HAMA behave if the vertices values are so big such that the
>> >> available RAM memory is not enough to contains all of the vertices?
>> >> >
>> >> > Regards
>> >>
>> >>
>> >>
>> >> --
>> >> Best Regards, Edward J. Yoon
>> >> @eddieyoon
>> >>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
>
>

Re: loading vertices into RAM

[Tommaso Teofili <to...@gmail.com>]

it's in hama-graph 0.6.3 [1].

Tommaso


[1] :
http://grepcode.com/file/repo1.maven.org$maven2@org.apache.hama$hama-graph@0.6.3@org$apache$hama$graph$OffHeapVerticesInfo.java


2014/1/15 Edward J. Yoon <ed...@apache.org>

> Today I tried to use off-heap vertices info but it throws Class not
> found exceptions. :/ Which jar need to be included to use it?
>
> On Wed, Jan 15, 2014 at 3:59 PM, Tommaso Teofili
> <to...@gmail.com> wrote:
> > and OffHeapVerticesInfo for loading vertices off heap, which is available
> > with 0.6.3 as well if I recall correctly.
> > Tommaso
> >
> >
> > 2014/1/15 Edward J. Yoon <ed...@apache.org>
> >
> >> There are few implementations.
> >>
> >>  - ListVerticesinfo: loads vertices into array list.
> >>  - MapVerticesinfo: loads vertices into tree map.
> >>  - DiskVerticesInfo: loads vertices into a local file.
> >>
> >> You can choose one of them by setting the "hama.graph.vertices.info"
> >> in job configuration.
> >>
> >>   > conf.setClass("hama.graph.vertices.info",
> >> "org.apache.hama.graph.ListVerticesInfo".
> >>
> >> With the latest 0.6.3 version, you can use only ListVerticesInfo.
> >> Please use the TRUNK.
> >>
> >>
> >> On Tue, Jan 14, 2014 at 11:18 PM, Ammar Sahib <am...@yahoo.com>
> >> wrote:
> >> > Hi
> >> >
> >> > According to the BSP model, the data is processed in the RAM and that
> is
> >> the reason why Pregel model is faster than the MapReduce (MapReduce
> >> writedown to disk). Can someone explains to me how to be sure that all
> the
> >> graph vertices are actually been loaded in RAM?
> >> >
> >> >
> >> > How would HAMA behave if the vertices values are so big such that the
> >> available RAM memory is not enough to contains all of the vertices?
> >> >
> >> > Regards
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>