You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hama.apache.org by "Edward J. Yoon" <ed...@apache.org> on 2012/07/10 12:48:45 UTC
Pagerank example doesn't work
I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3 tasks per each
node (bsp.child.java.opts = Xmx3512m), tried to run PageRank on
Wikipedia dataset introduced at
http://wiki.apache.org/hama/WriteHamaGraphFile
Doesn't work, the cause of job fail error is different every time.
NullPointerException at loadVertices(), ERROR bsp.BSPTask: Started
pinging to groom, ..., etc.
Has anyone tried this?
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Pagerank example doesn't work
Posted by Thomas Jungblut <th...@gmail.com>.
Great :)
I made a tutorial back then:
http://codingwiththomas.blogspot.de/2011/04/profiling-apache-hama-bsps.html
2012/8/30 Edward J. Yoon <ed...@apache.org>
> Yes, I'll try it.
>
> On Thu, Aug 30, 2012 at 10:38 PM, Thomas Jungblut
> <th...@gmail.com> wrote:
> > Still has some overhead, the wiki dataset I benched were just 1gb.
> > Wonder where our hotspots are, did you get your yourkit license to
> profile?
> >
> > 2012/8/30 Edward J. Yoon <ed...@apache.org>
> >
> >> My settings and envs were like this:
> >>
> >> * child.java.opts = -Xmx1024m
> >> * tasks.num per node = 5
> >> * total tasks = 85
> >> * network env = infiniband 40G
> >> * physical nodes = 17
> >> * hadoop-cdh-0.20
> >> * zookeeper-cdh
> >>
> >> Finished in 127 secs.
> >>
> >> On Thu, Aug 30, 2012 at 10:27 PM, Thomas Jungblut
> >> <th...@gmail.com> wrote:
> >> > Yeah, lots of ram ;))
> >> >
> >> > 2012/8/30 Edward J. Yoon <ed...@apache.org>
> >> >
> >> >> > It runs locally in under 120s on my computer
> >> >>
> >> >> Haha, single machine?
> >> >>
> >> >> On Thu, Aug 30, 2012 at 10:25 PM, Thomas Jungblut
> >> >> <th...@gmail.com> wrote:
> >> >> > It runs locally in under 120s on my computer so I assume that this
> is
> >> >> > correct.
> >> >> >
> >> >> > 2012/8/30 Edward J. Yoon <ed...@apache.org>
> >> >> >
> >> >> >> Just two minutes were needed to process wikipedia dataset. Is this
> >> >> normal?
> >> >> >>
> >> >> >> On Tue, Jul 10, 2012 at 8:17 PM, Thomas Jungblut
> >> >> >> <th...@gmail.com> wrote:
> >> >> >> > http://snap.stanford.edu/data/
> >> >> >> >
> >> >> >> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
> >> >> >> >
> >> >> >> >> Just wondering, do you know more large and downloadable data
> set?
> >> >> >> >>
> >> >> >> >> On Tue, Jul 10, 2012 at 8:01 PM, Edward J. Yoon <
> >> >> edwardyoon@apache.org>
> >> >> >> >> wrote:
> >> >> >> >> > Oh... thanks. now I can guess what it is.
> >> >> >> >> >
> >> >> >> >> > On Tue, Jul 10, 2012 at 7:57 PM, Thomas Jungblut
> >> >> >> >> > <th...@gmail.com> wrote:
> >> >> >> >> >> Yes, the example graph is not complete thus resulting in
> NPEs.
> >> >> >> >> >>
> >> >> >> >> >> 2012/7/10 Edward J. Yoon <ed...@apache.org>
> >> >> >> >> >>
> >> >> >> >> >>> Nope, should I set to true?
> >> >> >> >> >>>
> >> >> >> >> >>> On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut
> >> >> >> >> >>> <th...@gmail.com> wrote:
> >> >> >> >> >>> > Set "hama.graph.repair" to true in conf?
> >> >> >> >> >>> >
> >> >> >> >> >>> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
> >> >> >> >> >>> >
> >> >> >> >> >>> >> I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3
> tasks
> >> per
> >> >> >> each
> >> >> >> >> >>> >> node (bsp.child.java.opts = Xmx3512m), tried to run
> >> PageRank
> >> >> on
> >> >> >> >> >>> >> Wikipedia dataset introduced at
> >> >> >> >> >>> >> http://wiki.apache.org/hama/WriteHamaGraphFile
> >> >> >> >> >>> >>
> >> >> >> >> >>> >> Doesn't work, the cause of job fail error is different
> >> every
> >> >> >> time.
> >> >> >> >> >>> >> NullPointerException at loadVertices(), ERROR
> bsp.BSPTask:
> >> >> >> Started
> >> >> >> >> >>> >> pinging to groom, ..., etc.
> >> >> >> >> >>> >>
> >> >> >> >> >>> >> Has anyone tried this?
> >> >> >> >> >>> >>
> >> >> >> >> >>> >> --
> >> >> >> >> >>> >> Best Regards, Edward J. Yoon
> >> >> >> >> >>> >> @eddieyoon
> >> >> >> >> >>> >>
> >> >> >> >> >>>
> >> >> >> >> >>>
> >> >> >> >> >>>
> >> >> >> >> >>> --
> >> >> >> >> >>> Best Regards, Edward J. Yoon
> >> >> >> >> >>> @eddieyoon
> >> >> >> >> >>>
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> > --
> >> >> >> >> > Best Regards, Edward J. Yoon
> >> >> >> >> > @eddieyoon
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> --
> >> >> >> >> Best Regards, Edward J. Yoon
> >> >> >> >> @eddieyoon
> >> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Best Regards, Edward J. Yoon
> >> >> >> @eddieyoon
> >> >> >>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Best Regards, Edward J. Yoon
> >> >> @eddieyoon
> >> >>
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
Re: Pagerank example doesn't work
Posted by "Edward J. Yoon" <ed...@apache.org>.
Yes, I'll try it.
On Thu, Aug 30, 2012 at 10:38 PM, Thomas Jungblut
<th...@gmail.com> wrote:
> Still has some overhead, the wiki dataset I benched were just 1gb.
> Wonder where our hotspots are, did you get your yourkit license to profile?
>
> 2012/8/30 Edward J. Yoon <ed...@apache.org>
>
>> My settings and envs were like this:
>>
>> * child.java.opts = -Xmx1024m
>> * tasks.num per node = 5
>> * total tasks = 85
>> * network env = infiniband 40G
>> * physical nodes = 17
>> * hadoop-cdh-0.20
>> * zookeeper-cdh
>>
>> Finished in 127 secs.
>>
>> On Thu, Aug 30, 2012 at 10:27 PM, Thomas Jungblut
>> <th...@gmail.com> wrote:
>> > Yeah, lots of ram ;))
>> >
>> > 2012/8/30 Edward J. Yoon <ed...@apache.org>
>> >
>> >> > It runs locally in under 120s on my computer
>> >>
>> >> Haha, single machine?
>> >>
>> >> On Thu, Aug 30, 2012 at 10:25 PM, Thomas Jungblut
>> >> <th...@gmail.com> wrote:
>> >> > It runs locally in under 120s on my computer so I assume that this is
>> >> > correct.
>> >> >
>> >> > 2012/8/30 Edward J. Yoon <ed...@apache.org>
>> >> >
>> >> >> Just two minutes were needed to process wikipedia dataset. Is this
>> >> normal?
>> >> >>
>> >> >> On Tue, Jul 10, 2012 at 8:17 PM, Thomas Jungblut
>> >> >> <th...@gmail.com> wrote:
>> >> >> > http://snap.stanford.edu/data/
>> >> >> >
>> >> >> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
>> >> >> >
>> >> >> >> Just wondering, do you know more large and downloadable data set?
>> >> >> >>
>> >> >> >> On Tue, Jul 10, 2012 at 8:01 PM, Edward J. Yoon <
>> >> edwardyoon@apache.org>
>> >> >> >> wrote:
>> >> >> >> > Oh... thanks. now I can guess what it is.
>> >> >> >> >
>> >> >> >> > On Tue, Jul 10, 2012 at 7:57 PM, Thomas Jungblut
>> >> >> >> > <th...@gmail.com> wrote:
>> >> >> >> >> Yes, the example graph is not complete thus resulting in NPEs.
>> >> >> >> >>
>> >> >> >> >> 2012/7/10 Edward J. Yoon <ed...@apache.org>
>> >> >> >> >>
>> >> >> >> >>> Nope, should I set to true?
>> >> >> >> >>>
>> >> >> >> >>> On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut
>> >> >> >> >>> <th...@gmail.com> wrote:
>> >> >> >> >>> > Set "hama.graph.repair" to true in conf?
>> >> >> >> >>> >
>> >> >> >> >>> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
>> >> >> >> >>> >
>> >> >> >> >>> >> I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3 tasks
>> per
>> >> >> each
>> >> >> >> >>> >> node (bsp.child.java.opts = Xmx3512m), tried to run
>> PageRank
>> >> on
>> >> >> >> >>> >> Wikipedia dataset introduced at
>> >> >> >> >>> >> http://wiki.apache.org/hama/WriteHamaGraphFile
>> >> >> >> >>> >>
>> >> >> >> >>> >> Doesn't work, the cause of job fail error is different
>> every
>> >> >> time.
>> >> >> >> >>> >> NullPointerException at loadVertices(), ERROR bsp.BSPTask:
>> >> >> Started
>> >> >> >> >>> >> pinging to groom, ..., etc.
>> >> >> >> >>> >>
>> >> >> >> >>> >> Has anyone tried this?
>> >> >> >> >>> >>
>> >> >> >> >>> >> --
>> >> >> >> >>> >> Best Regards, Edward J. Yoon
>> >> >> >> >>> >> @eddieyoon
>> >> >> >> >>> >>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>> --
>> >> >> >> >>> Best Regards, Edward J. Yoon
>> >> >> >> >>> @eddieyoon
>> >> >> >> >>>
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > Best Regards, Edward J. Yoon
>> >> >> >> > @eddieyoon
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Best Regards, Edward J. Yoon
>> >> >> >> @eddieyoon
>> >> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Best Regards, Edward J. Yoon
>> >> >> @eddieyoon
>> >> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Best Regards, Edward J. Yoon
>> >> @eddieyoon
>> >>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Pagerank example doesn't work
Posted by Thomas Jungblut <th...@gmail.com>.
Still has some overhead, the wiki dataset I benched were just 1gb.
Wonder where our hotspots are, did you get your yourkit license to profile?
2012/8/30 Edward J. Yoon <ed...@apache.org>
> My settings and envs were like this:
>
> * child.java.opts = -Xmx1024m
> * tasks.num per node = 5
> * total tasks = 85
> * network env = infiniband 40G
> * physical nodes = 17
> * hadoop-cdh-0.20
> * zookeeper-cdh
>
> Finished in 127 secs.
>
> On Thu, Aug 30, 2012 at 10:27 PM, Thomas Jungblut
> <th...@gmail.com> wrote:
> > Yeah, lots of ram ;))
> >
> > 2012/8/30 Edward J. Yoon <ed...@apache.org>
> >
> >> > It runs locally in under 120s on my computer
> >>
> >> Haha, single machine?
> >>
> >> On Thu, Aug 30, 2012 at 10:25 PM, Thomas Jungblut
> >> <th...@gmail.com> wrote:
> >> > It runs locally in under 120s on my computer so I assume that this is
> >> > correct.
> >> >
> >> > 2012/8/30 Edward J. Yoon <ed...@apache.org>
> >> >
> >> >> Just two minutes were needed to process wikipedia dataset. Is this
> >> normal?
> >> >>
> >> >> On Tue, Jul 10, 2012 at 8:17 PM, Thomas Jungblut
> >> >> <th...@gmail.com> wrote:
> >> >> > http://snap.stanford.edu/data/
> >> >> >
> >> >> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
> >> >> >
> >> >> >> Just wondering, do you know more large and downloadable data set?
> >> >> >>
> >> >> >> On Tue, Jul 10, 2012 at 8:01 PM, Edward J. Yoon <
> >> edwardyoon@apache.org>
> >> >> >> wrote:
> >> >> >> > Oh... thanks. now I can guess what it is.
> >> >> >> >
> >> >> >> > On Tue, Jul 10, 2012 at 7:57 PM, Thomas Jungblut
> >> >> >> > <th...@gmail.com> wrote:
> >> >> >> >> Yes, the example graph is not complete thus resulting in NPEs.
> >> >> >> >>
> >> >> >> >> 2012/7/10 Edward J. Yoon <ed...@apache.org>
> >> >> >> >>
> >> >> >> >>> Nope, should I set to true?
> >> >> >> >>>
> >> >> >> >>> On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut
> >> >> >> >>> <th...@gmail.com> wrote:
> >> >> >> >>> > Set "hama.graph.repair" to true in conf?
> >> >> >> >>> >
> >> >> >> >>> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
> >> >> >> >>> >
> >> >> >> >>> >> I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3 tasks
> per
> >> >> each
> >> >> >> >>> >> node (bsp.child.java.opts = Xmx3512m), tried to run
> PageRank
> >> on
> >> >> >> >>> >> Wikipedia dataset introduced at
> >> >> >> >>> >> http://wiki.apache.org/hama/WriteHamaGraphFile
> >> >> >> >>> >>
> >> >> >> >>> >> Doesn't work, the cause of job fail error is different
> every
> >> >> time.
> >> >> >> >>> >> NullPointerException at loadVertices(), ERROR bsp.BSPTask:
> >> >> Started
> >> >> >> >>> >> pinging to groom, ..., etc.
> >> >> >> >>> >>
> >> >> >> >>> >> Has anyone tried this?
> >> >> >> >>> >>
> >> >> >> >>> >> --
> >> >> >> >>> >> Best Regards, Edward J. Yoon
> >> >> >> >>> >> @eddieyoon
> >> >> >> >>> >>
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>> --
> >> >> >> >>> Best Regards, Edward J. Yoon
> >> >> >> >>> @eddieyoon
> >> >> >> >>>
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> > --
> >> >> >> > Best Regards, Edward J. Yoon
> >> >> >> > @eddieyoon
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Best Regards, Edward J. Yoon
> >> >> >> @eddieyoon
> >> >> >>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Best Regards, Edward J. Yoon
> >> >> @eddieyoon
> >> >>
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
Re: Pagerank example doesn't work
Posted by "Edward J. Yoon" <ed...@apache.org>.
My settings and envs were like this:
* child.java.opts = -Xmx1024m
* tasks.num per node = 5
* total tasks = 85
* network env = infiniband 40G
* physical nodes = 17
* hadoop-cdh-0.20
* zookeeper-cdh
Finished in 127 secs.
On Thu, Aug 30, 2012 at 10:27 PM, Thomas Jungblut
<th...@gmail.com> wrote:
> Yeah, lots of ram ;))
>
> 2012/8/30 Edward J. Yoon <ed...@apache.org>
>
>> > It runs locally in under 120s on my computer
>>
>> Haha, single machine?
>>
>> On Thu, Aug 30, 2012 at 10:25 PM, Thomas Jungblut
>> <th...@gmail.com> wrote:
>> > It runs locally in under 120s on my computer so I assume that this is
>> > correct.
>> >
>> > 2012/8/30 Edward J. Yoon <ed...@apache.org>
>> >
>> >> Just two minutes were needed to process wikipedia dataset. Is this
>> normal?
>> >>
>> >> On Tue, Jul 10, 2012 at 8:17 PM, Thomas Jungblut
>> >> <th...@gmail.com> wrote:
>> >> > http://snap.stanford.edu/data/
>> >> >
>> >> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
>> >> >
>> >> >> Just wondering, do you know more large and downloadable data set?
>> >> >>
>> >> >> On Tue, Jul 10, 2012 at 8:01 PM, Edward J. Yoon <
>> edwardyoon@apache.org>
>> >> >> wrote:
>> >> >> > Oh... thanks. now I can guess what it is.
>> >> >> >
>> >> >> > On Tue, Jul 10, 2012 at 7:57 PM, Thomas Jungblut
>> >> >> > <th...@gmail.com> wrote:
>> >> >> >> Yes, the example graph is not complete thus resulting in NPEs.
>> >> >> >>
>> >> >> >> 2012/7/10 Edward J. Yoon <ed...@apache.org>
>> >> >> >>
>> >> >> >>> Nope, should I set to true?
>> >> >> >>>
>> >> >> >>> On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut
>> >> >> >>> <th...@gmail.com> wrote:
>> >> >> >>> > Set "hama.graph.repair" to true in conf?
>> >> >> >>> >
>> >> >> >>> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
>> >> >> >>> >
>> >> >> >>> >> I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3 tasks per
>> >> each
>> >> >> >>> >> node (bsp.child.java.opts = Xmx3512m), tried to run PageRank
>> on
>> >> >> >>> >> Wikipedia dataset introduced at
>> >> >> >>> >> http://wiki.apache.org/hama/WriteHamaGraphFile
>> >> >> >>> >>
>> >> >> >>> >> Doesn't work, the cause of job fail error is different every
>> >> time.
>> >> >> >>> >> NullPointerException at loadVertices(), ERROR bsp.BSPTask:
>> >> Started
>> >> >> >>> >> pinging to groom, ..., etc.
>> >> >> >>> >>
>> >> >> >>> >> Has anyone tried this?
>> >> >> >>> >>
>> >> >> >>> >> --
>> >> >> >>> >> Best Regards, Edward J. Yoon
>> >> >> >>> >> @eddieyoon
>> >> >> >>> >>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> --
>> >> >> >>> Best Regards, Edward J. Yoon
>> >> >> >>> @eddieyoon
>> >> >> >>>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Best Regards, Edward J. Yoon
>> >> >> > @eddieyoon
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Best Regards, Edward J. Yoon
>> >> >> @eddieyoon
>> >> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Best Regards, Edward J. Yoon
>> >> @eddieyoon
>> >>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Pagerank example doesn't work
Posted by Thomas Jungblut <th...@gmail.com>.
Yeah, lots of ram ;))
2012/8/30 Edward J. Yoon <ed...@apache.org>
> > It runs locally in under 120s on my computer
>
> Haha, single machine?
>
> On Thu, Aug 30, 2012 at 10:25 PM, Thomas Jungblut
> <th...@gmail.com> wrote:
> > It runs locally in under 120s on my computer so I assume that this is
> > correct.
> >
> > 2012/8/30 Edward J. Yoon <ed...@apache.org>
> >
> >> Just two minutes were needed to process wikipedia dataset. Is this
> normal?
> >>
> >> On Tue, Jul 10, 2012 at 8:17 PM, Thomas Jungblut
> >> <th...@gmail.com> wrote:
> >> > http://snap.stanford.edu/data/
> >> >
> >> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
> >> >
> >> >> Just wondering, do you know more large and downloadable data set?
> >> >>
> >> >> On Tue, Jul 10, 2012 at 8:01 PM, Edward J. Yoon <
> edwardyoon@apache.org>
> >> >> wrote:
> >> >> > Oh... thanks. now I can guess what it is.
> >> >> >
> >> >> > On Tue, Jul 10, 2012 at 7:57 PM, Thomas Jungblut
> >> >> > <th...@gmail.com> wrote:
> >> >> >> Yes, the example graph is not complete thus resulting in NPEs.
> >> >> >>
> >> >> >> 2012/7/10 Edward J. Yoon <ed...@apache.org>
> >> >> >>
> >> >> >>> Nope, should I set to true?
> >> >> >>>
> >> >> >>> On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut
> >> >> >>> <th...@gmail.com> wrote:
> >> >> >>> > Set "hama.graph.repair" to true in conf?
> >> >> >>> >
> >> >> >>> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
> >> >> >>> >
> >> >> >>> >> I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3 tasks per
> >> each
> >> >> >>> >> node (bsp.child.java.opts = Xmx3512m), tried to run PageRank
> on
> >> >> >>> >> Wikipedia dataset introduced at
> >> >> >>> >> http://wiki.apache.org/hama/WriteHamaGraphFile
> >> >> >>> >>
> >> >> >>> >> Doesn't work, the cause of job fail error is different every
> >> time.
> >> >> >>> >> NullPointerException at loadVertices(), ERROR bsp.BSPTask:
> >> Started
> >> >> >>> >> pinging to groom, ..., etc.
> >> >> >>> >>
> >> >> >>> >> Has anyone tried this?
> >> >> >>> >>
> >> >> >>> >> --
> >> >> >>> >> Best Regards, Edward J. Yoon
> >> >> >>> >> @eddieyoon
> >> >> >>> >>
> >> >> >>>
> >> >> >>>
> >> >> >>>
> >> >> >>> --
> >> >> >>> Best Regards, Edward J. Yoon
> >> >> >>> @eddieyoon
> >> >> >>>
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Best Regards, Edward J. Yoon
> >> >> > @eddieyoon
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Best Regards, Edward J. Yoon
> >> >> @eddieyoon
> >> >>
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
Re: Pagerank example doesn't work
Posted by "Edward J. Yoon" <ed...@apache.org>.
> It runs locally in under 120s on my computer
Haha, single machine?
On Thu, Aug 30, 2012 at 10:25 PM, Thomas Jungblut
<th...@gmail.com> wrote:
> It runs locally in under 120s on my computer so I assume that this is
> correct.
>
> 2012/8/30 Edward J. Yoon <ed...@apache.org>
>
>> Just two minutes were needed to process wikipedia dataset. Is this normal?
>>
>> On Tue, Jul 10, 2012 at 8:17 PM, Thomas Jungblut
>> <th...@gmail.com> wrote:
>> > http://snap.stanford.edu/data/
>> >
>> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
>> >
>> >> Just wondering, do you know more large and downloadable data set?
>> >>
>> >> On Tue, Jul 10, 2012 at 8:01 PM, Edward J. Yoon <ed...@apache.org>
>> >> wrote:
>> >> > Oh... thanks. now I can guess what it is.
>> >> >
>> >> > On Tue, Jul 10, 2012 at 7:57 PM, Thomas Jungblut
>> >> > <th...@gmail.com> wrote:
>> >> >> Yes, the example graph is not complete thus resulting in NPEs.
>> >> >>
>> >> >> 2012/7/10 Edward J. Yoon <ed...@apache.org>
>> >> >>
>> >> >>> Nope, should I set to true?
>> >> >>>
>> >> >>> On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut
>> >> >>> <th...@gmail.com> wrote:
>> >> >>> > Set "hama.graph.repair" to true in conf?
>> >> >>> >
>> >> >>> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
>> >> >>> >
>> >> >>> >> I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3 tasks per
>> each
>> >> >>> >> node (bsp.child.java.opts = Xmx3512m), tried to run PageRank on
>> >> >>> >> Wikipedia dataset introduced at
>> >> >>> >> http://wiki.apache.org/hama/WriteHamaGraphFile
>> >> >>> >>
>> >> >>> >> Doesn't work, the cause of job fail error is different every
>> time.
>> >> >>> >> NullPointerException at loadVertices(), ERROR bsp.BSPTask:
>> Started
>> >> >>> >> pinging to groom, ..., etc.
>> >> >>> >>
>> >> >>> >> Has anyone tried this?
>> >> >>> >>
>> >> >>> >> --
>> >> >>> >> Best Regards, Edward J. Yoon
>> >> >>> >> @eddieyoon
>> >> >>> >>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> --
>> >> >>> Best Regards, Edward J. Yoon
>> >> >>> @eddieyoon
>> >> >>>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Best Regards, Edward J. Yoon
>> >> > @eddieyoon
>> >>
>> >>
>> >>
>> >> --
>> >> Best Regards, Edward J. Yoon
>> >> @eddieyoon
>> >>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Pagerank example doesn't work
Posted by Thomas Jungblut <th...@gmail.com>.
It runs locally in under 120s on my computer so I assume that this is
correct.
2012/8/30 Edward J. Yoon <ed...@apache.org>
> Just two minutes were needed to process wikipedia dataset. Is this normal?
>
> On Tue, Jul 10, 2012 at 8:17 PM, Thomas Jungblut
> <th...@gmail.com> wrote:
> > http://snap.stanford.edu/data/
> >
> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
> >
> >> Just wondering, do you know more large and downloadable data set?
> >>
> >> On Tue, Jul 10, 2012 at 8:01 PM, Edward J. Yoon <ed...@apache.org>
> >> wrote:
> >> > Oh... thanks. now I can guess what it is.
> >> >
> >> > On Tue, Jul 10, 2012 at 7:57 PM, Thomas Jungblut
> >> > <th...@gmail.com> wrote:
> >> >> Yes, the example graph is not complete thus resulting in NPEs.
> >> >>
> >> >> 2012/7/10 Edward J. Yoon <ed...@apache.org>
> >> >>
> >> >>> Nope, should I set to true?
> >> >>>
> >> >>> On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut
> >> >>> <th...@gmail.com> wrote:
> >> >>> > Set "hama.graph.repair" to true in conf?
> >> >>> >
> >> >>> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
> >> >>> >
> >> >>> >> I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3 tasks per
> each
> >> >>> >> node (bsp.child.java.opts = Xmx3512m), tried to run PageRank on
> >> >>> >> Wikipedia dataset introduced at
> >> >>> >> http://wiki.apache.org/hama/WriteHamaGraphFile
> >> >>> >>
> >> >>> >> Doesn't work, the cause of job fail error is different every
> time.
> >> >>> >> NullPointerException at loadVertices(), ERROR bsp.BSPTask:
> Started
> >> >>> >> pinging to groom, ..., etc.
> >> >>> >>
> >> >>> >> Has anyone tried this?
> >> >>> >>
> >> >>> >> --
> >> >>> >> Best Regards, Edward J. Yoon
> >> >>> >> @eddieyoon
> >> >>> >>
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Best Regards, Edward J. Yoon
> >> >>> @eddieyoon
> >> >>>
> >> >
> >> >
> >> >
> >> > --
> >> > Best Regards, Edward J. Yoon
> >> > @eddieyoon
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
Re: Pagerank example doesn't work
Posted by "Edward J. Yoon" <ed...@apache.org>.
Just two minutes were needed to process wikipedia dataset. Is this normal?
On Tue, Jul 10, 2012 at 8:17 PM, Thomas Jungblut
<th...@gmail.com> wrote:
> http://snap.stanford.edu/data/
>
> 2012/7/10 Edward J. Yoon <ed...@apache.org>
>
>> Just wondering, do you know more large and downloadable data set?
>>
>> On Tue, Jul 10, 2012 at 8:01 PM, Edward J. Yoon <ed...@apache.org>
>> wrote:
>> > Oh... thanks. now I can guess what it is.
>> >
>> > On Tue, Jul 10, 2012 at 7:57 PM, Thomas Jungblut
>> > <th...@gmail.com> wrote:
>> >> Yes, the example graph is not complete thus resulting in NPEs.
>> >>
>> >> 2012/7/10 Edward J. Yoon <ed...@apache.org>
>> >>
>> >>> Nope, should I set to true?
>> >>>
>> >>> On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut
>> >>> <th...@gmail.com> wrote:
>> >>> > Set "hama.graph.repair" to true in conf?
>> >>> >
>> >>> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
>> >>> >
>> >>> >> I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3 tasks per each
>> >>> >> node (bsp.child.java.opts = Xmx3512m), tried to run PageRank on
>> >>> >> Wikipedia dataset introduced at
>> >>> >> http://wiki.apache.org/hama/WriteHamaGraphFile
>> >>> >>
>> >>> >> Doesn't work, the cause of job fail error is different every time.
>> >>> >> NullPointerException at loadVertices(), ERROR bsp.BSPTask: Started
>> >>> >> pinging to groom, ..., etc.
>> >>> >>
>> >>> >> Has anyone tried this?
>> >>> >>
>> >>> >> --
>> >>> >> Best Regards, Edward J. Yoon
>> >>> >> @eddieyoon
>> >>> >>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Best Regards, Edward J. Yoon
>> >>> @eddieyoon
>> >>>
>> >
>> >
>> >
>> > --
>> > Best Regards, Edward J. Yoon
>> > @eddieyoon
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Pagerank example doesn't work
Posted by Thomas Jungblut <th...@gmail.com>.
http://snap.stanford.edu/data/
2012/7/10 Edward J. Yoon <ed...@apache.org>
> Just wondering, do you know more large and downloadable data set?
>
> On Tue, Jul 10, 2012 at 8:01 PM, Edward J. Yoon <ed...@apache.org>
> wrote:
> > Oh... thanks. now I can guess what it is.
> >
> > On Tue, Jul 10, 2012 at 7:57 PM, Thomas Jungblut
> > <th...@gmail.com> wrote:
> >> Yes, the example graph is not complete thus resulting in NPEs.
> >>
> >> 2012/7/10 Edward J. Yoon <ed...@apache.org>
> >>
> >>> Nope, should I set to true?
> >>>
> >>> On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut
> >>> <th...@gmail.com> wrote:
> >>> > Set "hama.graph.repair" to true in conf?
> >>> >
> >>> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
> >>> >
> >>> >> I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3 tasks per each
> >>> >> node (bsp.child.java.opts = Xmx3512m), tried to run PageRank on
> >>> >> Wikipedia dataset introduced at
> >>> >> http://wiki.apache.org/hama/WriteHamaGraphFile
> >>> >>
> >>> >> Doesn't work, the cause of job fail error is different every time.
> >>> >> NullPointerException at loadVertices(), ERROR bsp.BSPTask: Started
> >>> >> pinging to groom, ..., etc.
> >>> >>
> >>> >> Has anyone tried this?
> >>> >>
> >>> >> --
> >>> >> Best Regards, Edward J. Yoon
> >>> >> @eddieyoon
> >>> >>
> >>>
> >>>
> >>>
> >>> --
> >>> Best Regards, Edward J. Yoon
> >>> @eddieyoon
> >>>
> >
> >
> >
> > --
> > Best Regards, Edward J. Yoon
> > @eddieyoon
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
Re: Pagerank example doesn't work
Posted by "Edward J. Yoon" <ed...@apache.org>.
Just wondering, do you know more large and downloadable data set?
On Tue, Jul 10, 2012 at 8:01 PM, Edward J. Yoon <ed...@apache.org> wrote:
> Oh... thanks. now I can guess what it is.
>
> On Tue, Jul 10, 2012 at 7:57 PM, Thomas Jungblut
> <th...@gmail.com> wrote:
>> Yes, the example graph is not complete thus resulting in NPEs.
>>
>> 2012/7/10 Edward J. Yoon <ed...@apache.org>
>>
>>> Nope, should I set to true?
>>>
>>> On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut
>>> <th...@gmail.com> wrote:
>>> > Set "hama.graph.repair" to true in conf?
>>> >
>>> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
>>> >
>>> >> I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3 tasks per each
>>> >> node (bsp.child.java.opts = Xmx3512m), tried to run PageRank on
>>> >> Wikipedia dataset introduced at
>>> >> http://wiki.apache.org/hama/WriteHamaGraphFile
>>> >>
>>> >> Doesn't work, the cause of job fail error is different every time.
>>> >> NullPointerException at loadVertices(), ERROR bsp.BSPTask: Started
>>> >> pinging to groom, ..., etc.
>>> >>
>>> >> Has anyone tried this?
>>> >>
>>> >> --
>>> >> Best Regards, Edward J. Yoon
>>> >> @eddieyoon
>>> >>
>>>
>>>
>>>
>>> --
>>> Best Regards, Edward J. Yoon
>>> @eddieyoon
>>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Pagerank example doesn't work
Posted by "Edward J. Yoon" <ed...@apache.org>.
Oh... thanks. now I can guess what it is.
On Tue, Jul 10, 2012 at 7:57 PM, Thomas Jungblut
<th...@gmail.com> wrote:
> Yes, the example graph is not complete thus resulting in NPEs.
>
> 2012/7/10 Edward J. Yoon <ed...@apache.org>
>
>> Nope, should I set to true?
>>
>> On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut
>> <th...@gmail.com> wrote:
>> > Set "hama.graph.repair" to true in conf?
>> >
>> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
>> >
>> >> I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3 tasks per each
>> >> node (bsp.child.java.opts = Xmx3512m), tried to run PageRank on
>> >> Wikipedia dataset introduced at
>> >> http://wiki.apache.org/hama/WriteHamaGraphFile
>> >>
>> >> Doesn't work, the cause of job fail error is different every time.
>> >> NullPointerException at loadVertices(), ERROR bsp.BSPTask: Started
>> >> pinging to groom, ..., etc.
>> >>
>> >> Has anyone tried this?
>> >>
>> >> --
>> >> Best Regards, Edward J. Yoon
>> >> @eddieyoon
>> >>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Pagerank example doesn't work
Posted by Thomas Jungblut <th...@gmail.com>.
Yes, the example graph is not complete thus resulting in NPEs.
2012/7/10 Edward J. Yoon <ed...@apache.org>
> Nope, should I set to true?
>
> On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut
> <th...@gmail.com> wrote:
> > Set "hama.graph.repair" to true in conf?
> >
> > 2012/7/10 Edward J. Yoon <ed...@apache.org>
> >
> >> I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3 tasks per each
> >> node (bsp.child.java.opts = Xmx3512m), tried to run PageRank on
> >> Wikipedia dataset introduced at
> >> http://wiki.apache.org/hama/WriteHamaGraphFile
> >>
> >> Doesn't work, the cause of job fail error is different every time.
> >> NullPointerException at loadVertices(), ERROR bsp.BSPTask: Started
> >> pinging to groom, ..., etc.
> >>
> >> Has anyone tried this?
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
Re: Pagerank example doesn't work
Posted by "Edward J. Yoon" <ed...@apache.org>.
Nope, should I set to true?
On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut
<th...@gmail.com> wrote:
> Set "hama.graph.repair" to true in conf?
>
> 2012/7/10 Edward J. Yoon <ed...@apache.org>
>
>> I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3 tasks per each
>> node (bsp.child.java.opts = Xmx3512m), tried to run PageRank on
>> Wikipedia dataset introduced at
>> http://wiki.apache.org/hama/WriteHamaGraphFile
>>
>> Doesn't work, the cause of job fail error is different every time.
>> NullPointerException at loadVertices(), ERROR bsp.BSPTask: Started
>> pinging to groom, ..., etc.
>>
>> Has anyone tried this?
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Pagerank example doesn't work
Posted by Thomas Jungblut <th...@gmail.com>.
Set "hama.graph.repair" to true in conf?
2012/7/10 Edward J. Yoon <ed...@apache.org>
> I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3 tasks per each
> node (bsp.child.java.opts = Xmx3512m), tried to run PageRank on
> Wikipedia dataset introduced at
> http://wiki.apache.org/hama/WriteHamaGraphFile
>
> Doesn't work, the cause of job fail error is different every time.
> NullPointerException at loadVertices(), ERROR bsp.BSPTask: Started
> pinging to groom, ..., etc.
>
> Has anyone tried this?
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>