Lazy unrolling of iterative algorithms

[Asterios Katsifodimos <as...@tu-berlin.de>]

Hello,

I would like to use the feature of lazy unrolling in order to run KMeans.

Would this be possible in he current API? Or should I use a lower level API
(what we used to call Nephele?)

Thanks,
Asterios

Re: Lazy unrolling of iterative algorithms

[Stephan Ewen <se...@apache.org>]

Hey!

The Java API does not yet support it, the JobGraph supports it, though.

In the runtime, it will still execute everything from the start if you
"attach" another rolled out version of the loop."

We should have that working in a few weeks.

What is your schedule? How fast do you need it?

Stephan


On Thu, Oct 30, 2014 at 11:43 AM, Asterios Katsifodimos <
asterios.katsifodimos@tu-berlin.de> wrote:

> Hello,
>
> I would like to use the feature of lazy unrolling in order to run KMeans.
>
> Would this be possible in he current API? Or should I use a lower level
> API (what we used to call Nephele?)
>
> Thanks,
> Asterios
>

Re: Lazy unrolling of iterative algorithms

[Asterios Katsifodimos <as...@tu-berlin.de>]

We woudl like to have asap actually.

Our deadline is on Thursday. However, I think I could potentially work on
the JobGraph.

What we would like to see is the effect of re-deploying a job (a la Spark)
on every iteration and measure it in large deployments (i.e. the overhead
of not persisting tasks on machines).

Do you think we could "simulate" this in an easy way?

Best,
Asterios





On Thursday, October 30, 2014, Stephan Ewen <se...@apache.org> wrote:

>  Hey!
>
>  The Java API does not yet support it, the JobGraph supports it, though.
>
>  In the runtime, it will still execute everything from the start if you
> "attach" another rolled out version of the loop."
>
>  We should have that working in a few weeks.
>
>  What is your schedule? How fast do you need it?
>
>  Stephan
>
>
> On Thu, Oct 30, 2014 at 11:43 AM, Asterios Katsifodimos <
> asterios.katsifodimos@tu-berlin.de> wrote:
>
>> Hello,
>>
>>  I would like to use the feature of lazy unrolling in order to run
>> KMeans.
>>
>>  Would this be possible in he current API? Or should I use a lower level
>> API (what we used to call Nephele?)
>>
>>  Thanks,
>> Asterios
>>
>
>