You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Soheil Pourbafrani <so...@gmail.com> on 2020/01/16 16:10:23 UTC

Read CSV file and and create customized field

Hi friends,
I'm going to read a CSV file that has 3 columns. I want the final loaded
datatype to have other columns inferred by that 3 columns. For example, I
would split the first column of the CSV file and create 3 new columns.
The problem is I did not find a straightforward approach for that. Here is
what I have so far:

env.readCsvFile("pathToCsv")
        .fieldDelimiter(",")
        .ignoreFirstLine()
        .ignoreInvalidLines()
        .type(String.class, String.class, String.class)
        .print();

So is there any way to show the readCSVFile how to split CSV records or I
should add an extra map function after loading CSV to create my desired
schema?

Thanks

Re: Read CSV file and and create customized field

Posted by Chesnay Schepler <ch...@apache.org>.

You should add an extra map function.

On 16/01/2020 17:10, Soheil Pourbafrani wrote:
> Hi friends,
> I'm going to read a CSV file that has 3 columns. I want the final 
> loaded datatype to have other columns inferred by that 3 columns. 
> For example, I would split the first column of the CSV file and create 
> 3 new columns.
> The problem is I did not find a straightforward approach for that. 
> Here is what I have so far:
> env.readCsvFile("pathToCsv")
>          .fieldDelimiter(",")
>          .ignoreFirstLine()
>          .ignoreInvalidLines()
>          .type(String.class, String.class, String.class)
>          .print();
> So is there any way to show the readCSVFile how to split CSV records 
> or I should add an extra map function after loading CSV to create my 
> desired schema?
>
> Thanks