The Iteration and Superstep

[Shuo Wang <ec...@gmail.com>]

Hi,

What is the difference between the *iterations* and the *supersteps* when
we run pagerank or sssp on HAMA?

wangshuo

Re: The Iteration and Superstep

[Suraj Menon <su...@apache.org>]

>From what I found while refactoring the code for GraphJobRunner in HAMA-632
is that roughly (not looking at edge cases):

if (aggregationRunner.isEnabled) /// If we have aggregators setup the
values would be aggregated by master in the extra superstep.
iterations = 2 * (number of supersteps after all vertices are loaded,
partitioned and repaired)
else
iterations = (number of supersteps after all vertices are loaded,
partitioned and repaired)

I think we can maintain the last execution completion time in
Vertex.compute to find the time for each iteration. Shouldn't this work?

-Suraj


On Thu, Jan 3, 2013 at 8:34 AM, Shuo Wang <ec...@gmail.com> wrote:

> But now we can't know the superstep is the *iteration* or the *
> additional superstep*.
>
> 2013/1/3 Edward J. Yoon <ed...@apache.org>
>
> > > Do you mean that the time of each *interation* equals the time of per *
> > > superstep*?
> >
> > Yes, each iteration process is a superstep of BSP program.
> >
> > On Thu, Jan 3, 2013 at 10:21 PM, Shuo Wang <ec...@gmail.com>
> > wrote:
> > > Do you mean that the time of each *interation* equals the time of per *
> > > superstep*?
> > >
> > > 2013/1/3 Edward J. Yoon <ed...@apache.org>
> > >
> > >> We don't logging execution time of each iteration yet.
> > >>
> > >> But, it's not difficult. You can get current time before and after
> > >> calling doSuperstep() method, and then calculate execution time of
> > >> each iteration. If you want to improve our project, Pls feel free to
> > >> create JIRA tickets and attach your patches. ;)
> > >>
> > >> On Thu, Jan 3, 2013 at 9:52 PM, Shuo Wang <ec...@gmail.com>
> > >> wrote:
> > >> > Year,I got it.But how can I kown the executed time per iteration?
> > >> >
> > >> > 2013/1/3 Edward J. Yoon <ed...@apache.org>
> > >> >
> > >> >> (Pregel-like) graph computations consist of a sequence of
> > *iterations*.
> > >> >>
> > >> >> To perform the graph job, internally, we also do additional global
> > >> >> synchronizations. For example, counts and synchronizes the number
> of
> > >> >> total vertices among Workers.
> > >> >>
> > >> >> So, the total *supersteps* of BSP program for graph job =
> > *iterations*
> > >> >> + additional supersteps.
> > >> >>
> > >> >> On Thu, Jan 3, 2013 at 6:44 PM, Shuo Wang <
> ecisp.wangshuo@gmail.com>
> > >> >> wrote:
> > >> >> > Hi,
> > >> >> >
> > >> >> > What is the difference between the *iterations* and the
> > *supersteps*
> > >> when
> > >> >> > we run pagerank or sssp on HAMA?
> > >> >> >
> > >> >> > wangshuo
> > >> >>
> > >> >>
> > >> >>
> > >> >> --
> > >> >> Best Regards, Edward J. Yoon
> > >> >> @eddieyoon
> > >> >>
> > >>
> > >>
> > >>
> > >> --
> > >> Best Regards, Edward J. Yoon
> > >> @eddieyoon
> > >>
> >
> >
> >
> > --
> > Best Regards, Edward J. Yoon
> > @eddieyoon
> >
>

Re: The Iteration and Superstep

[Shuo Wang <ec...@gmail.com>]

But now we can't know the superstep is the *iteration* or the *
additional superstep*.

2013/1/3 Edward J. Yoon <ed...@apache.org>

> > Do you mean that the time of each *interation* equals the time of per *
> > superstep*?
>
> Yes, each iteration process is a superstep of BSP program.
>
> On Thu, Jan 3, 2013 at 10:21 PM, Shuo Wang <ec...@gmail.com>
> wrote:
> > Do you mean that the time of each *interation* equals the time of per *
> > superstep*?
> >
> > 2013/1/3 Edward J. Yoon <ed...@apache.org>
> >
> >> We don't logging execution time of each iteration yet.
> >>
> >> But, it's not difficult. You can get current time before and after
> >> calling doSuperstep() method, and then calculate execution time of
> >> each iteration. If you want to improve our project, Pls feel free to
> >> create JIRA tickets and attach your patches. ;)
> >>
> >> On Thu, Jan 3, 2013 at 9:52 PM, Shuo Wang <ec...@gmail.com>
> >> wrote:
> >> > Year,I got it.But how can I kown the executed time per iteration?
> >> >
> >> > 2013/1/3 Edward J. Yoon <ed...@apache.org>
> >> >
> >> >> (Pregel-like) graph computations consist of a sequence of
> *iterations*.
> >> >>
> >> >> To perform the graph job, internally, we also do additional global
> >> >> synchronizations. For example, counts and synchronizes the number of
> >> >> total vertices among Workers.
> >> >>
> >> >> So, the total *supersteps* of BSP program for graph job =
> *iterations*
> >> >> + additional supersteps.
> >> >>
> >> >> On Thu, Jan 3, 2013 at 6:44 PM, Shuo Wang <ec...@gmail.com>
> >> >> wrote:
> >> >> > Hi,
> >> >> >
> >> >> > What is the difference between the *iterations* and the
> *supersteps*
> >> when
> >> >> > we run pagerank or sssp on HAMA?
> >> >> >
> >> >> > wangshuo
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Best Regards, Edward J. Yoon
> >> >> @eddieyoon
> >> >>
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>

Re: The Iteration and Superstep

["Edward J. Yoon" <ed...@apache.org>]

> Do you mean that the time of each *interation* equals the time of per *
> superstep*?

Yes, each iteration process is a superstep of BSP program.

On Thu, Jan 3, 2013 at 10:21 PM, Shuo Wang <ec...@gmail.com> wrote:
> Do you mean that the time of each *interation* equals the time of per *
> superstep*?
>
> 2013/1/3 Edward J. Yoon <ed...@apache.org>
>
>> We don't logging execution time of each iteration yet.
>>
>> But, it's not difficult. You can get current time before and after
>> calling doSuperstep() method, and then calculate execution time of
>> each iteration. If you want to improve our project, Pls feel free to
>> create JIRA tickets and attach your patches. ;)
>>
>> On Thu, Jan 3, 2013 at 9:52 PM, Shuo Wang <ec...@gmail.com>
>> wrote:
>> > Year,I got it.But how can I kown the executed time per iteration?
>> >
>> > 2013/1/3 Edward J. Yoon <ed...@apache.org>
>> >
>> >> (Pregel-like) graph computations consist of a sequence of *iterations*.
>> >>
>> >> To perform the graph job, internally, we also do additional global
>> >> synchronizations. For example, counts and synchronizes the number of
>> >> total vertices among Workers.
>> >>
>> >> So, the total *supersteps* of BSP program for graph job = *iterations*
>> >> + additional supersteps.
>> >>
>> >> On Thu, Jan 3, 2013 at 6:44 PM, Shuo Wang <ec...@gmail.com>
>> >> wrote:
>> >> > Hi,
>> >> >
>> >> > What is the difference between the *iterations* and the *supersteps*
>> when
>> >> > we run pagerank or sssp on HAMA?
>> >> >
>> >> > wangshuo
>> >>
>> >>
>> >>
>> >> --
>> >> Best Regards, Edward J. Yoon
>> >> @eddieyoon
>> >>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Re: The Iteration and Superstep

[Shuo Wang <ec...@gmail.com>]

Do you mean that the time of each *interation* equals the time of per *
superstep*?

2013/1/3 Edward J. Yoon <ed...@apache.org>

> We don't logging execution time of each iteration yet.
>
> But, it's not difficult. You can get current time before and after
> calling doSuperstep() method, and then calculate execution time of
> each iteration. If you want to improve our project, Pls feel free to
> create JIRA tickets and attach your patches. ;)
>
> On Thu, Jan 3, 2013 at 9:52 PM, Shuo Wang <ec...@gmail.com>
> wrote:
> > Year,I got it.But how can I kown the executed time per iteration?
> >
> > 2013/1/3 Edward J. Yoon <ed...@apache.org>
> >
> >> (Pregel-like) graph computations consist of a sequence of *iterations*.
> >>
> >> To perform the graph job, internally, we also do additional global
> >> synchronizations. For example, counts and synchronizes the number of
> >> total vertices among Workers.
> >>
> >> So, the total *supersteps* of BSP program for graph job = *iterations*
> >> + additional supersteps.
> >>
> >> On Thu, Jan 3, 2013 at 6:44 PM, Shuo Wang <ec...@gmail.com>
> >> wrote:
> >> > Hi,
> >> >
> >> > What is the difference between the *iterations* and the *supersteps*
> when
> >> > we run pagerank or sssp on HAMA?
> >> >
> >> > wangshuo
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>

Re: The Iteration and Superstep

["Edward J. Yoon" <ed...@apache.org>]

We don't logging execution time of each iteration yet.

But, it's not difficult. You can get current time before and after
calling doSuperstep() method, and then calculate execution time of
each iteration. If you want to improve our project, Pls feel free to
create JIRA tickets and attach your patches. ;)

On Thu, Jan 3, 2013 at 9:52 PM, Shuo Wang <ec...@gmail.com> wrote:
> Year,I got it.But how can I kown the executed time per iteration?
>
> 2013/1/3 Edward J. Yoon <ed...@apache.org>
>
>> (Pregel-like) graph computations consist of a sequence of *iterations*.
>>
>> To perform the graph job, internally, we also do additional global
>> synchronizations. For example, counts and synchronizes the number of
>> total vertices among Workers.
>>
>> So, the total *supersteps* of BSP program for graph job = *iterations*
>> + additional supersteps.
>>
>> On Thu, Jan 3, 2013 at 6:44 PM, Shuo Wang <ec...@gmail.com>
>> wrote:
>> > Hi,
>> >
>> > What is the difference between the *iterations* and the *supersteps* when
>> > we run pagerank or sssp on HAMA?
>> >
>> > wangshuo
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Re: The Iteration and Superstep

[Shuo Wang <ec...@gmail.com>]

Year,I got it.But how can I kown the executed time per iteration?

2013/1/3 Edward J. Yoon <ed...@apache.org>

> (Pregel-like) graph computations consist of a sequence of *iterations*.
>
> To perform the graph job, internally, we also do additional global
> synchronizations. For example, counts and synchronizes the number of
> total vertices among Workers.
>
> So, the total *supersteps* of BSP program for graph job = *iterations*
> + additional supersteps.
>
> On Thu, Jan 3, 2013 at 6:44 PM, Shuo Wang <ec...@gmail.com>
> wrote:
> > Hi,
> >
> > What is the difference between the *iterations* and the *supersteps* when
> > we run pagerank or sssp on HAMA?
> >
> > wangshuo
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>

Re: The Iteration and Superstep

["Edward J. Yoon" <ed...@apache.org>]

(Pregel-like) graph computations consist of a sequence of *iterations*.

To perform the graph job, internally, we also do additional global
synchronizations. For example, counts and synchronizes the number of
total vertices among Workers.

So, the total *supersteps* of BSP program for graph job = *iterations*
+ additional supersteps.

On Thu, Jan 3, 2013 at 6:44 PM, Shuo Wang <ec...@gmail.com> wrote:
> Hi,
>
> What is the difference between the *iterations* and the *supersteps* when
> we run pagerank or sssp on HAMA?
>
> wangshuo



-- 
Best Regards, Edward J. Yoon
@eddieyoon