You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Abdul Salam Shaikh <ab...@gmail.com> on 2017/04/28 09:18:37 UTC

Writing the results of the stream onto a CSV File

Hi,

I am trying to write the results of my stream into a CSV format using the
following code and it has compilation issues:

DataStream<DetectorStatistics> objectStream = windowedStream.flatMap(new
WindowObjectStreamTransformer());
        objectStream.writeAsText("H:\\data.csv", new
TextFormatter<DetectorStatistics>() {
            public String format (DetectorStatistics value) {
                return value.getDetectorId() + " ," +
value.getActualPhaseTime() ;
            }
        });

What am I doing wrong here ? Or is there an alternative way to write
records onto a CSV file ? 

Thanks!

Re: Writing the results of the stream onto a CSV File

Posted by Till Rohrmann <tr...@apache.org>.

Hi Abdul,

the DataStream#writeAsText does not support a TextFormatter as argument.
You either have to implement your own OutputFormat and calling
DataStream#writeUsingOutputFormat or as Fabio recommended simply use
DataStream#writeAsCsv.

Cheers,
Till


On Fri, Apr 28, 2017 at 11:46 AM, Fábio Dias <fa...@gmail.com> wrote:

> Hi,
>
> Instead of use writeAsText you have a writeAsCsv https://ci.apache.
> org/projects/flink/flink-docs-master/api/java/org/apache/
> flink/streaming/api/datastream/DataStream.html
>
> You can use just with the string path (like you have) or you can use the
> overwrite flag if it suit your needs.
>
> Best Regards,
> Fábio Dias.
>
> Abdul Salam Shaikh <ab...@gmail.com> escreveu no dia sexta,
> 28/04/2017 às 10:18:
>
>> Hi,
>>
>> I am trying to write the results of my stream into a CSV format using the
>> following code and it has compilation issues:
>>
>> DataStream<DetectorStatistics> objectStream = windowedStream.flatMap(new
>> WindowObjectStreamTransformer());
>>         objectStream.writeAsText("H:\\data.csv", new TextFormatter<DetectorStatistics>()
>> {
>>             public String format (DetectorStatistics value) {
>>                 return value.getDetectorId() + " ," +
>> value.getActualPhaseTime() ;
>>             }
>>         });
>>
>> What am I doing wrong here ? Or is there an alternative way to write
>> records onto a CSV file ? 
>>
>> Thanks!
>>
>>

Re: Writing the results of the stream onto a CSV File

Posted by Fábio Dias <fa...@gmail.com>.

Hi,

Instead of use writeAsText you have a writeAsCsv
https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/streaming/api/datastream/DataStream.html

You can use just with the string path (like you have) or you can use the
overwrite flag if it suit your needs.

Best Regards,
Fábio Dias.

Abdul Salam Shaikh <ab...@gmail.com> escreveu no dia sexta,
28/04/2017 às 10:18:

> Hi,
>
> I am trying to write the results of my stream into a CSV format using the
> following code and it has compilation issues:
>
> DataStream<DetectorStatistics> objectStream = windowedStream.flatMap(new
> WindowObjectStreamTransformer());
>         objectStream.writeAsText("H:\\data.csv", new
> TextFormatter<DetectorStatistics>() {
>             public String format (DetectorStatistics value) {
>                 return value.getDetectorId() + " ," +
> value.getActualPhaseTime() ;
>             }
>         });
>
> What am I doing wrong here ? Or is there an alternative way to write
> records onto a CSV file ? 
>
> Thanks!
>
>