You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by 杰 <25...@qq.com> on 2016/10/21 06:37:45 UTC
Re: Unable to load CSV using differenent delimiter like "|"(Pipe) and";" (Semicolon) etc
hi, Harmeet
For problem 1:
u need to keep that column name is also delimited by '|',
that is , name|description|salary|age|dob
For problem 2:
in Load command, change "DELIMITER"=";" to "DELIMITER"="\;" ,
because ";" is usually means the end, so need to be escaped.
thanks
Jay
------------------ Original ------------------
From: "Harmeet";<ha...@gmail.com>;
Date: Fri, Oct 21, 2016 02:23 PM
To: "dev"<de...@carbondata.incubator.apache.org>;
Subject: Unable to load CSV using differenent delimiter like "|"(Pipe) and";" (Semicolon) etc
Hey Team,
I am trying to load data from csv using different delimiters like "|"(Pipe)
and ";" (Semicolon) etc. But the system gave me an different errors
according to delimiter. The example as below:
*create table one (name string, description string, salary double, age int,
dob timestamp) stored by 'carbondata';*
*1. "|" (Pipe Delimeter)*
deliandquote.csv File >>
name, description, salary, age, dob
tammy| 'my name'| 900000| 22| 19/10/2019
0: jdbc:hive2://127.0.0.1:10000> load data local inpath
'hdfs://localhost:54310/home/harmeet/deliandquote.csv' into table one
OPTIONS("DELIMITER"="|", 'QUOTECHAR'="'");
Error: java.lang.Exception: DataLoad failure: CSV File provided is not
proper. Column names in schema and csv header are not same. CSVFile Name :
deliandquote.csv (state=,code=0)
*2. ";" (Semicolon)*
semiandquote.csv File >>
0: jdbc:hive2://127.0.0.1:10000> load data local inpath
'hdfs://localhost:54310/home/harmeet/semiandquote.csv' into table one
OPTIONS("DELIMITER"=";", 'QUOTECHAR'="'");
Error: org.apache.spark.sql.AnalysisException: missing EOF at 'OPTIONS' near
'one'; line 1 pos 93 (state=,code=0)
I am not getting, why the different errors are shown, if we are not
supporting other characters. Or this is the defect of carbondata ?
--
View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Unable-to-load-CSV-using-differenent-delimiter-like-Pipe-and-Semicolon-etc-tp2140.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com.
Re: Unable to load CSV using differenent delimiter like
"|"(Pipe)and";" (Semicolon) etc
Posted by Harmeet <ha...@gmail.com>.
Hey Jay,
I am getting still same error. My code is up to date. I am creating new
database and new table for testing. Following are details:
*>> Create database *
0: jdbc:hive2://127.0.0.1:10000> create database needb;
+---------+--+
| result |
+---------+--+
+---------+--+
No rows selected (6.227 seconds)
*>> use database*
0: jdbc:hive2://127.0.0.1:10000> use needb;
+---------+--+
| result |
+---------+--+
+---------+--+
No rows selected (0.081 seconds)
*>> Create table*
0: jdbc:hive2://127.0.0.1:10000> create table one(name string, description
string, salary double, age int, dob timestamp) stored by 'carbondata';
+---------+--+
| Result |
+---------+--+
+---------+--+
No rows selected (1.297 seconds)
*>> CSV File Data*
name; description; salary; age; dob
tammy; my name; 900000; 22; 19/10/2019
*>> Load Data *
0: jdbc:hive2://127.0.0.1:10000> load data inpath
'hdfs://localhost:54310/home/harmeet/semiandquote.csv' into table one
options('DELIMITER'="\;");
Error: org.apache.spark.sql.AnalysisException: missing EOF at 'options' near
'one'; line 1 pos 87 (state=,code=0)
I am still not figure out, what is missing ?
--
View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Unable-to-load-CSV-using-differenent-delimiter-like-Pipe-and-Semicolon-etc-tp2140p2163.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com.
Re: Unable to load CSV using differenent delimiter like "|"(Pipe)and";" (Semicolon) etc
Posted by 杰 <25...@qq.com>.
hi, Harmeet
not reproduce. pls provide more details.
reagrds
Jay
------------------ Original ------------------
From: "Harmeet";<ha...@gmail.com>;
Date: Fri, Oct 21, 2016 02:51 PM
To: "dev"<de...@carbondata.incubator.apache.org>;
Subject: Re: Unable to load CSV using differenent delimiter like "|"(Pipe)and";" (Semicolon) etc
Hey Jay, Thanks for your reply.
First case with Pipe symbols working fine, But still in second case i am
getting the same error. Below is my query :
csv file >>
name; description; salary; age; dob
tammy; 'my name'; 900000; 22; 19/10/2019
0: jdbc:hive2://127.0.0.1:10000> load data local inpath
'hdfs://localhost:54310/home/harmeet/semiandquote.csv' into table one
OPTIONS("DELIMITER"="\;", 'QUOTECHAR'="'");
Error: org.apache.spark.sql.AnalysisException: missing EOF at 'OPTIONS' near
'one'; line 1 pos 93 (state=,code=0)
Please correnct if something getting wrong.
--
View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Unable-to-load-CSV-using-differenent-delimiter-like-Pipe-and-Semicolon-etc-tp2140p2144.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com.
Re: Unable to load CSV using differenent delimiter like "|"(Pipe)
and";" (Semicolon) etc
Posted by Harmeet <ha...@gmail.com>.
Hey Jay, Thanks for your reply.
First case with Pipe symbols working fine, But still in second case i am
getting the same error. Below is my query :
csv file >>
name; description; salary; age; dob
tammy; 'my name'; 900000; 22; 19/10/2019
0: jdbc:hive2://127.0.0.1:10000> load data local inpath
'hdfs://localhost:54310/home/harmeet/semiandquote.csv' into table one
OPTIONS("DELIMITER"="\;", 'QUOTECHAR'="'");
Error: org.apache.spark.sql.AnalysisException: missing EOF at 'OPTIONS' near
'one'; line 1 pos 93 (state=,code=0)
Please correnct if something getting wrong.
--
View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Unable-to-load-CSV-using-differenent-delimiter-like-Pipe-and-Semicolon-etc-tp2140p2144.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com.