wait for "writeAsText" to finish

[Lydia <ic...@googlemail.com>]

Hi, 

I have a program that contains a preprocessing with Flink Objects and at the end writes the result with „result.writeAsText(„...“)“.
After that I call a method that is basically a MapReduce-Job (actually only a Map-Job) which depends on the written file.

So what is the smartest way to delay the execution of the Map-Job until the file is written completely?
Right now I I am doing it the following way:

val written = result.writeAsText(„…“)
if(written.getDataSet.count() > 0){ ...do Map-Job...}

Thanks in advance!
Best regards,
Lydia

Re: wait for "writeAsText" to finish

[Chesnay Schepler <ch...@apache.org>]

Hello,

Since ExecutionEnvironment#execute() blocks until the job is finished 
you should be able to just do this:

data.writeAsText();
env.execute();
{ do Map-Job }

Note that your current solution is wrong, as it translates to this:

DataSet result = ...
result.writeAsText();
if (result.count() > 0){ ... do Map-Job ... }

Regards,
Chesnay

On 20.04.2017 09:46, Lydia wrote:
> Hi,
>
> I have a program that contains a preprocessing with Flink Objects and at the end writes the result with \u201eresult.writeAsText(\u201e...\u201c)\u201c.
> After that I call a method that is basically a MapReduce-Job (actually only a Map-Job) which depends on the written file.
>
> So what is the smartest way to delay the execution of the Map-Job until the file is written completely?
> Right now I I am doing it the following way:
>
> val written = result.writeAsText(\u201e\u2026\u201c)
> if(written.getDataSet.count() > 0){ ...do Map-Job...}
>
> Thanks in advance!
> Best regards,
> Lydia