Issue with writeAsText() to S3 bucket

["Nguyen, Michael" <Mi...@T-Mobile.com>]

Hello all,

I am running into issues at the moment trying to print my DataStreams to an S3 bucket using writeAsText(“s3://bucket/result.json”) in my Flink job. I used print() on the same DataStream and I see the output I am looking for in standard output. I first confirm that my datastream has data by looking at the standard output, then I cancel my Flink job. After cancelling the job, result.json only gets created in my S3 bucket some of the time. It does not always gets created, but I do confirm that I do have standard output.

I understand writeAsText() should be used for debugging purposes only according to docs, but I’m just curious as to why I can’t get writeAsText() to always work every time I cancel my job.

Thank you for your help,
Michael