Intermediate solution set of delta iteration

[Mengqi Yang <me...@gmail.com>]

Hi all,

Are there any approaches here I could get intermediate solution sets from
every delta iteration? I tried union but the compiler gave me the error: 
Exception in thread "main"
org.apache.flink.api.common.InvalidProgramException: Error: The only
operations allowed on the solution set are Join and CoGroup.


Best,
Mengqi 



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Intermediate-solution-set-of-delta-iteration-tp5656.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: Intermediate solution set of delta iteration

[Vasiliki Kalavri <va...@gmail.com>]

Hi Mengqi,

if what you are trying to do is output the solution set of every iteration,
before the iteration has finished, then that is not possible.
i.e. you can not output the solution set to a sink or another operator
during the iteration.

However, you can add elements to the solution set and grow the dataset as
the iteration proceeds and retrieve it after it is finished.
Currently, you cannot union with the solution set, but a CoGroup could give
you the desired result.

Alternatively, you might want to look at bulk iterations (Iterate
abstraction) [1]. With these you can produce a completely new dataset after
every iteration.

I hope that helps!
-Vasia.

[1]:
https://ci.apache.org/projects/flink/flink-docs-master/apis/batch/iterations.html

On 20 March 2016 at 18:12, Mengqi Yang <me...@gmail.com> wrote:

> Hi all,
>
> Are there any approaches here I could get intermediate solution sets from
> every delta iteration? I tried union but the compiler gave me the error:
> Exception in thread "main"
> org.apache.flink.api.common.InvalidProgramException: Error: The only
> operations allowed on the solution set are Join and CoGroup.
>
>
> Best,
> Mengqi
>
>
>
> --
> View this message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Intermediate-solution-set-of-delta-iteration-tp5656.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>