You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by narasimha <sw...@gmail.com> on 2020/12/03 16:45:03 UTC
How to parse list values in csv file
Hi,
Getting below error when trying to read a csv file, one of the field is
list tupe
Can someone help if fixing the issue
jobmanager_1 | Caused by: java.lang.IllegalArgumentException: The type
'java.util.List' is not supported for the CSV input format.
jobmanager_1 | at
org.apache.flink.api.common.io.GenericCsvInputFormat.setFieldsGeneric(GenericCsvInputFormat.java:289)
~[flink-dist_2.11-1.11.2.jar:1.11.2]
jobmanager_1 | at
org.apache.flink.api.java.io.RowCsvInputFormat.<init>(RowCsvInputFormat.java:67)
~[flink-dist_2.11-1.11.2.jar:1.11.2]
jobmanager_1 | at
org.apache.flink.api.java.io.RowCsvInputFormat.<init>(RowCsvInputFormat.java:83)
~[flink-dist_2.11-1.11.2.jar:1.11.2]
jobmanager_1 | at
org.apache.flink.api.java.io.RowCsvInputFormat.<init>(RowCsvInputFormat.java:87)
~[flink-dist_2.11-1.11.2.jar:1.11.2]
--
A.Narasimha Swamy
Re: How to parse list values in csv file
Posted by Yun Gao <yu...@aliyun.com>.
Glad to hear that you solved this issue!
Best,
Yun------------------------------------------------------------------
Sender:narasimha<sw...@gmail.com>
Date:2020/12/06 21:35:33
Recipient:Yun Gao<yu...@aliyun.com>
Cc:user<us...@flink.apache.org>
Theme:Re: How to parse list values in csv file
thanks for you email.
Translated csv to JSON, read it as a plain text file and then processed to objects.
It solved my use case.
On Fri, Dec 4, 2020 at 12:24 PM Yun Gao <yu...@aliyun.com> wrote:
Hi,
The CSV only supports the types listed in [1] and must use the types in this list, thus for other types some kind of workaround is needed, like first parsed as string and parsed again later in the program.
Best,
Yun
[1] https://github.com/apache/flink/blob/e10e548feb2bedf54c3863bbd49ed4f9140546cf/flink-core/src/main/java/org/apache/flink/types/parser/FieldParser.java#L287
------------------Original Mail ------------------
Sender:narasimha <sw...@gmail.com>
Send Date:Fri Dec 4 00:45:53 2020
Recipients:user <us...@flink.apache.org>
Subject:How to parse list values in csv file
Hi,
Getting below error when trying to read a csv file, one of the field is list tupe
Can someone help if fixing the issue
jobmanager_1 | Caused by: java.lang.IllegalArgumentException: The type 'java.util.List' is not supported for the CSV input format.
jobmanager_1 | at org.apache.flink.api.common.io.GenericCsvInputFormat.setFieldsGeneric(GenericCsvInputFormat.java:289) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
jobmanager_1 | at org.apache.flink.api.java.io.RowCsvInputFormat.<init>(RowCsvInputFormat.java:67) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
jobmanager_1 | at org.apache.flink.api.java.io.RowCsvInputFormat.<init>(RowCsvInputFormat.java:83) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
jobmanager_1 | at org.apache.flink.api.java.io.RowCsvInputFormat.<init>(RowCsvInputFormat.java:87) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
--
A.Narasimha Swamy
--
A.Narasimha Swamy
Re: How to parse list values in csv file
Posted by narasimha <sw...@gmail.com>.
thanks for you email.
Translated csv to JSON, read it as a plain text file and then processed to
objects.
It solved my use case.
On Fri, Dec 4, 2020 at 12:24 PM Yun Gao <yu...@aliyun.com> wrote:
>
> Hi,
>
> The CSV only supports the types listed in [1] and must use the types
> in this list, thus for other types some kind of workaround is needed, like
> first parsed as string and parsed again later in the program.
>
> Best,
> Yun
>
>
>
> [1]
> https://github.com/apache/flink/blob/e10e548feb2bedf54c3863bbd49ed4f9140546cf/flink-core/src/main/java/org/apache/flink/types/parser/FieldParser.java#L287
>
>
> ------------------Original Mail ------------------
> *Sender:*narasimha <sw...@gmail.com>
> *Send Date:*Fri Dec 4 00:45:53 2020
> *Recipients:*user <us...@flink.apache.org>
> *Subject:*How to parse list values in csv file
>
>> Hi,
>>
>> Getting below error when trying to read a csv file, one of the field is
>> list tupe
>>
>> Can someone help if fixing the issue
>>
>> jobmanager_1 | Caused by: java.lang.IllegalArgumentException: The type
>> 'java.util.List' is not supported for the CSV input format.
>>
>> jobmanager_1 | at
>> org.apache.flink.api.common.io.GenericCsvInputFormat.setFieldsGeneric(GenericCsvInputFormat.java:289)
>> ~[flink-dist_2.11-1.11.2.jar:1.11.2]
>>
>> jobmanager_1 | at
>> org.apache.flink.api.java.io.RowCsvInputFormat.<init>(RowCsvInputFormat.java:67)
>> ~[flink-dist_2.11-1.11.2.jar:1.11.2]
>>
>> jobmanager_1 | at
>> org.apache.flink.api.java.io.RowCsvInputFormat.<init>(RowCsvInputFormat.java:83)
>> ~[flink-dist_2.11-1.11.2.jar:1.11.2]
>>
>> jobmanager_1 | at
>> org.apache.flink.api.java.io.RowCsvInputFormat.<init>(RowCsvInputFormat.java:87)
>> ~[flink-dist_2.11-1.11.2.jar:1.11.2]
>>
>> --
>> A.Narasimha Swamy
>>
>
--
A.Narasimha Swamy
Re: How to parse list values in csv file
Posted by Yun Gao <yu...@aliyun.com>.
Hi,
The CSV only supports the types listed in [1] and must use the types in this list, thus for other types some kind of workaround is needed, like first parsed as string and parsed again later in the program.
Best,
Yun
[1] https://github.com/apache/flink/blob/e10e548feb2bedf54c3863bbd49ed4f9140546cf/flink-core/src/main/java/org/apache/flink/types/parser/FieldParser.java#L287
------------------Original Mail ------------------
Sender:narasimha <sw...@gmail.com>
Send Date:Fri Dec 4 00:45:53 2020
Recipients:user <us...@flink.apache.org>
Subject:How to parse list values in csv file
Hi,
Getting below error when trying to read a csv file, one of the field is list tupe
Can someone help if fixing the issue
jobmanager_1 | Caused by: java.lang.IllegalArgumentException: The type 'java.util.List' is not supported for the CSV input format.
jobmanager_1 | at org.apache.flink.api.common.io.GenericCsvInputFormat.setFieldsGeneric(GenericCsvInputFormat.java:289) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
jobmanager_1 | at org.apache.flink.api.java.io.RowCsvInputFormat.<init>(RowCsvInputFormat.java:67) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
jobmanager_1 | at org.apache.flink.api.java.io.RowCsvInputFormat.<init>(RowCsvInputFormat.java:83) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
jobmanager_1 | at org.apache.flink.api.java.io.RowCsvInputFormat.<init>(RowCsvInputFormat.java:87) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
--
A.Narasimha Swamy