Spark Thrift Server - Skip header when load data from local file

[kumar r <ku...@gmail.com>]

Hi,

I want to load data from csv file to Spark table using Spark thrift server.
When i load, header(first line) should be ignored.

I have used tblproperties("skip.header.line.count"="1") option. But its not
working and first line also included.

Below is spark sql query i have tried.

create table tabname(id string,name string) row format delimited fields
terminated by ',' tblproperties("skip.header.line.count"="1");
load data local inpath 'tabname.csv' overwrite into table tabname;

How can i achieve this? Is there any other solution or workaround.

Re: Spark Thrift Server - Skip header when load data from local file

[ayan guha <gu...@gmail.com>]

I doubt you can do that. Create a staging table and then insert into main
table after filtering the header
On Wed, 15 Feb 2017 at 4:01 pm, kumar r <ku...@gmail.com> wrote:

> Hi,
>
> I want to load data from csv file to Spark table using Spark thrift
> server. When i load, header(first line) should be ignored.
>
> I have used tblproperties("skip.header.line.count"="1") option. But its
> not working and first line also included.
>
> Below is spark sql query i have tried.
>
> create table tabname(id string,name string) row format delimited fields
> terminated by ',' tblproperties("skip.header.line.count"="1");
> load data local inpath 'tabname.csv' overwrite into table tabname;
>
> How can i achieve this? Is there any other solution or workaround.
>
-- 
Best Regards,
Ayan Guha