You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Lydia Ickler <ic...@googlemail.com> on 2016/03/31 09:39:48 UTC

wait until BulkIteration finishes

Hi all,

is there a way to tell the program that it should wait until the BulkIteration finishes before the rest of the program is executed?

Best regards,
Lydia

Re: wait until BulkIteration finishes

Posted by Lydia Ickler <ic...@googlemail.com>.

Hi,

thanks for your reply!
Could you please give me an example for the close() step? I can’t find an example online only for open().
There I can „save“ my new result?

Best regards,
Lydia


> Am 31.03.2016 um 18:16 schrieb Stephan Ewen <se...@apache.org>:
> 
> Hi Lydia!
> 
> The same function instances (for example MapFunction objects) are used across all supersteps. If, for example, you store something in a HashMap inside some MapFunction, you can access that in the next iteration superstep.
> 
> You can figure out when a superstep finished and when the next superstep starts, by overriding "open()" and "close()" from the RichFunction interface.
> 
> Stephan
> 
> 
> On Thu, Mar 31, 2016 at 4:45 PM, Till Rohrmann <trohrmann@apache.org <ma...@apache.org>> wrote:
> I think I don't completely understand your question.
> 
> On Thu, Mar 31, 2016 at 4:40 PM, Lydia Ickler <icklerly@googlemail.com <ma...@googlemail.com>> wrote:
> Hi Till, 
> 
> thanks for your reply!
> Is there a way to store intermediate results of the bulk iteration to use then in the next iteration except the data set one sends already by default?
> 
> Best regards, 
> Lydia
> 
> 
>> Am 31.03.2016 um 12:01 schrieb Till Rohrmann <trohrmann@apache.org <ma...@apache.org>>:
>> 
>> Hi Lydia,
>> 
>> all downstream operators which depend on the bulk iteration will wait implicitly until data from the iteration operator is available.
>> 
>> Cheers,
>> Till
>> 
>> On Thu, Mar 31, 2016 at 9:39 AM, Lydia Ickler <icklerly@googlemail.com <ma...@googlemail.com>> wrote:
>> Hi all,
>> 
>> is there a way to tell the program that it should wait until the BulkIteration finishes before the rest of the program is executed?
>> 
>> Best regards,
>> Lydia
>> 
> 
> 
>

Re: wait until BulkIteration finishes

Posted by Stephan Ewen <se...@apache.org>.

Hi Lydia!

The same function instances (for example MapFunction objects) are used
across all supersteps. If, for example, you store something in a HashMap
inside some MapFunction, you can access that in the next iteration
superstep.

You can figure out when a superstep finished and when the next superstep
starts, by overriding "open()" and "close()" from the RichFunction
interface.

Stephan

On Thu, Mar 31, 2016 at 4:45 PM, Till Rohrmann <tr...@apache.org> wrote:

> I think I don't completely understand your question.
>
> On Thu, Mar 31, 2016 at 4:40 PM, Lydia Ickler <ic...@googlemail.com>
> wrote:
>
>> Hi Till,
>>
>> thanks for your reply!
>> Is there a way to store intermediate results of the bulk iteration to use
>> then in the next iteration except the data set one sends already by default?
>>
>> Best regards,
>> Lydia
>>
>>
>> Am 31.03.2016 um 12:01 schrieb Till Rohrmann <tr...@apache.org>:
>>
>> Hi Lydia,
>>
>> all downstream operators which depend on the bulk iteration will wait
>> implicitly until data from the iteration operator is available.
>>
>> Cheers,
>> Till
>>
>> On Thu, Mar 31, 2016 at 9:39 AM, Lydia Ickler <ic...@googlemail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> is there a way to tell the program that it should wait until the
>>> BulkIteration finishes before the rest of the program is executed?
>>>
>>> Best regards,
>>> Lydia
>>
>>
>>
>>
>

Re: wait until BulkIteration finishes

Posted by Till Rohrmann <tr...@apache.org>.

I think I don't completely understand your question.

On Thu, Mar 31, 2016 at 4:40 PM, Lydia Ickler <ic...@googlemail.com>
wrote:

> Hi Till,
>
> thanks for your reply!
> Is there a way to store intermediate results of the bulk iteration to use
> then in the next iteration except the data set one sends already by default?
>
> Best regards,
> Lydia
>
>
> Am 31.03.2016 um 12:01 schrieb Till Rohrmann <tr...@apache.org>:
>
> Hi Lydia,
>
> all downstream operators which depend on the bulk iteration will wait
> implicitly until data from the iteration operator is available.
>
> Cheers,
> Till
>
> On Thu, Mar 31, 2016 at 9:39 AM, Lydia Ickler <ic...@googlemail.com>
> wrote:
>
>> Hi all,
>>
>> is there a way to tell the program that it should wait until the
>> BulkIteration finishes before the rest of the program is executed?
>>
>> Best regards,
>> Lydia
>
>
>
>

Re: wait until BulkIteration finishes

Posted by Lydia Ickler <ic...@googlemail.com>.

Hi Till, 

thanks for your reply!
Is there a way to store intermediate results of the bulk iteration to use then in the next iteration except the data set one sends already by default?

Best regards, 
Lydia


> Am 31.03.2016 um 12:01 schrieb Till Rohrmann <tr...@apache.org>:
> 
> Hi Lydia,
> 
> all downstream operators which depend on the bulk iteration will wait implicitly until data from the iteration operator is available.
> 
> Cheers,
> Till
> 
> On Thu, Mar 31, 2016 at 9:39 AM, Lydia Ickler <icklerly@googlemail.com <ma...@googlemail.com>> wrote:
> Hi all,
> 
> is there a way to tell the program that it should wait until the BulkIteration finishes before the rest of the program is executed?
> 
> Best regards,
> Lydia
>

Re: wait until BulkIteration finishes

Posted by Till Rohrmann <tr...@apache.org>.

Hi Lydia,

all downstream operators which depend on the bulk iteration will wait
implicitly until data from the iteration operator is available.

Cheers,
Till

On Thu, Mar 31, 2016 at 9:39 AM, Lydia Ickler <ic...@googlemail.com>
wrote:

> Hi all,
>
> is there a way to tell the program that it should wait until the
> BulkIteration finishes before the rest of the program is executed?
>
> Best regards,
> Lydia