You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Fabian Hueske <fh...@gmail.com> on 2017/02/06 11:20:33 UTC
Re: Bug in Table api CsvTableSink
Hi Flavio,
I checked the records on the current master and the CsvTableSink seemed to
work fine.
However, I had some issues when converting the DataSet[Row] into a table.
You have to make sure that the TypeInformation for DataSet[Row] is
RowTypeInfo and not GenericType[Row].
Can you check the type of your myDataSet?
Best, Fabian
2017-01-31 17:51 GMT+01:00 Flavio Pompermaier <po...@okkam.it>:
> These 2 rows if converted to Row[] of Strings should cause the problem:
>
> http://www.aaa.it/xxx/v/10002780063,,,,t/0000000/1,f/
> 10001957530,cf/13,cpva/77,cf/13,,sit/A2046X,strp/408,
> 10921957530,,1,5,1,2013-01-04T15:02:25,5,,10002780063,
> XXXXXXX,1,,3,,,2013-01-04T15:02:25,XXXXXXX,XXXXXXX,13,2013-01-04T15:02:25
> http://www.aaa.it/xxx/v/10002780063,,,,t/0000000/1,f/
> 10004002060,cf/3,cpva/7,cf/3,f/10164002060,sit/A15730L,
> strp/408,10164002060,10164002060,2,7,1,2008-05-29T11:47:35,1,,10002780063,
> XXXXXXX,1,,0,,,2008-05-29T11:47:35,XXXXXXX,XXXXXXX,3,2008-05-29T11:47:35
>
> Best,
> Flavio
>
> On Tue, Jan 31, 2017 at 4:51 PM, Flavio Pompermaier <po...@okkam.it>
> wrote:
>
>> I hope to have time to write a test program :)
>> Otherwise I hope someone else could give it a try in the meantime..
>>
>> Best,
>> Flavio
>>
>> On Tue, Jan 31, 2017 at 4:49 PM, Fabian Hueske <fh...@gmail.com> wrote:
>>
>>> Hi Flavio,
>>>
>>> I do not remember that such a bug was fixed. Maybe by chance, but I
>>> guess not.
>>> Can you open a JIRA and maybe provide input data to reproduce the
>>> problem?
>>>
>>> Thank you,
>>> Fabian
>>>
>>> 2017-01-31 16:25 GMT+01:00 Flavio Pompermaier <po...@okkam.it>:
>>>
>>>> Hi to all,
>>>> I'm trying to read from a db and then writing to a csv.
>>>> In my code I do the following:
>>>>
>>>> tableEnv.fromDataSet(myDataSet).writeToSink(new
>>>> CsvTableSink(csvOutputDir, fieldDelim));
>>>>
>>>> If I use fieldDelim= "," everything is Ok, if I use "\t" some tab is
>>>> not printed correctly...
>>>> PS: myDataSet is a dataset of 32 String fields.
>>>>
>>>> Is is something that has been fixed in Flink > 1.1.1?
>>>>
>>>> Best,
>>>> Flavio
>>>>
>>>>
>>
>
Re: Bug in Table api CsvTableSink
Posted by Flavio Pompermaier <po...@okkam.it>.
The error was just caused by a cut-and-paste on the shell instead of
opening directly the file so bug solved :)
On Mon, Feb 6, 2017 at 12:20 PM, Fabian Hueske <fh...@gmail.com> wrote:
> Hi Flavio,
>
> I checked the records on the current master and the CsvTableSink seemed to
> work fine.
> However, I had some issues when converting the DataSet[Row] into a table.
> You have to make sure that the TypeInformation for DataSet[Row] is
> RowTypeInfo and not GenericType[Row].
>
> Can you check the type of your myDataSet?
>
> Best, Fabian
>
> 2017-01-31 17:51 GMT+01:00 Flavio Pompermaier <po...@okkam.it>:
>
>> These 2 rows if converted to Row[] of Strings should cause the problem:
>>
>> http://www.aaa.it/xxx/v/10002780063,,,,t/0000000/1,f/1000195
>> 7530,cf/13,cpva/77,cf/13,,sit/A2046X,strp/408,10921957530,,1
>> ,5,1,2013-01-04T15:02:25,5,,10002780063,XXXXXXX,1,,3,,,2013-
>> 01-04T15:02:25,XXXXXXX,XXXXXXX,13,2013-01-04T15:02:25
>> http://www.aaa.it/xxx/v/10002780063,,,,t/0000000/1,f/1000400
>> 2060,cf/3,cpva/7,cf/3,f/10164002060,sit/A15730L,strp/408,
>> 10164002060,10164002060,2,7,1,2008-05-29T11:47:35,1,,1000278
>> 0063,XXXXXXX,1,,0,,,2008-05-29T11:47:35,XXXXXXX,XXXXXXX,3,
>> 2008-05-29T11:47:35
>>
>> Best,
>> Flavio
>>
>> On Tue, Jan 31, 2017 at 4:51 PM, Flavio Pompermaier <pompermaier@okkam.it
>> > wrote:
>>
>>> I hope to have time to write a test program :)
>>> Otherwise I hope someone else could give it a try in the meantime..
>>>
>>> Best,
>>> Flavio
>>>
>>> On Tue, Jan 31, 2017 at 4:49 PM, Fabian Hueske <fh...@gmail.com>
>>> wrote:
>>>
>>>> Hi Flavio,
>>>>
>>>> I do not remember that such a bug was fixed. Maybe by chance, but I
>>>> guess not.
>>>> Can you open a JIRA and maybe provide input data to reproduce the
>>>> problem?
>>>>
>>>> Thank you,
>>>> Fabian
>>>>
>>>> 2017-01-31 16:25 GMT+01:00 Flavio Pompermaier <po...@okkam.it>:
>>>>
>>>>> Hi to all,
>>>>> I'm trying to read from a db and then writing to a csv.
>>>>> In my code I do the following:
>>>>>
>>>>> tableEnv.fromDataSet(myDataSet).writeToSink(new
>>>>> CsvTableSink(csvOutputDir, fieldDelim));
>>>>>
>>>>> If I use fieldDelim= "," everything is Ok, if I use "\t" some tab is
>>>>> not printed correctly...
>>>>> PS: myDataSet is a dataset of 32 String fields.
>>>>>
>>>>> Is is something that has been fixed in Flink > 1.1.1?
>>>>>
>>>>> Best,
>>>>> Flavio
>>>>>
>>>>>
>>>
>>