You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by Dharmesh Guna <dg...@vmware.com> on 2016/07/14 05:52:39 UTC
Issue in Bulkload when Table name contains '-' (hyphen)
Dear All,
I am facing issue in bulk loading from csv files into Phoenix when my Phoenix table contains '-' (hyphen) in table name.
I am using Phoenix version 4.7.0. Phoenix supports '-' (hyphen) in table name as per naming conventions. [a-zA-Z_0-9-.]
If I try to bulk load using below command it fails and gives below error.
sudo -u hadoop HADOOP_CLASSPATH=/usr/lib/hbase/hbase-protocol.jar:/usr/lib/hbase/conf/ hadoop jar /usr/lib/phoenix/phoenix-client.jar org.apache.phoenix.mapreduce.CsvBulkLoadTool -Dfs.permissions.umask-mode=000 -t 'DMG-TEST' --input "/user/test/1/DMG-TEST.csv" -d $'\t'
So ultimately it removes any single or double quotes around table name before upserting which causes error 601.
2016-07-14 05:39:07,380 WARN [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system already initialized!
2016-07-14 05:39:08,944 INFO [main] org.apache.phoenix.util.UpsertExecutor: Upserting SQL data with UPSERT INTO DMG-TEST ("APPLICATIONID", "0"."PACKAGEID") VALUES (?, ?)
2016-07-14 05:39:08,945 INFO [main] org.apache.hadoop.mapred.MapTask: Starting flush of map output
2016-07-14 05:39:08,952 INFO [main] org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.snappy]
2016-07-14 05:39:08,959 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: org.apache.phoenix.exception.PhoenixParserException: ERROR 601 (42P00): Syntax error. Encountered "-" at line 1, column 17.
at org.apache.phoenix.util.UpsertExecutor.createStatement(UpsertExecutor.java:83)
at org.apache.phoenix.util.UpsertExecutor.<init>(UpsertExecutor.java:94)
at org.apache.phoenix.util.csv.CsvUpsertExecutor.<init>(CsvUpsertExecutor.java:63)
at org.apache.phoenix.mapreduce.CsvToKeyValueMapper.buildUpsertExecutor(CsvToKeyValueMapper.java:85)
at org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.setup(FormatToBytesWritableMapper.java:142)
at org.apache.phoenix.mapreduce.CsvToKeyValueMapper.setup(CsvToKeyValueMapper.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:796)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.phoenix.exception.PhoenixParserException: ERROR 601 (42P00): Syntax error. Encountered "-" at line 1, column 17.
at org.apache.phoenix.exception.PhoenixParserException.newException(PhoenixParserException.java:33)
at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:111)
at org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1185)
at org.apache.phoenix.jdbc.PhoenixStatement.parseStatement(PhoenixStatement.java:1268)
at org.apache.phoenix.jdbc.PhoenixPreparedStatement.<init>(PhoenixPreparedStatement.java:94)
at org.apache.phoenix.jdbc.PhoenixConnection.prepareStatement(PhoenixConnection.java:715)
at org.apache.phoenix.util.UpsertExecutor.createStatement(UpsertExecutor.java:81)
... 13 more
Caused by: NoViableAltException(94@[])
at org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:4723)
at org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:775)
at org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:500)
at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
... 18 more
Thank you!!
Regards,
:: Dharmesh Guna
:: Development | Custom Reports
:: Contact | 961.199.87.52 | Ext. | 130 3264
Fwd: Issue in Bulkload when Table name contains '-' (hyphen)
Posted by "rajeshbabu@apache.org" <ch...@gmail.com>.
This is a bug. You can raise an issue Dharmesh,
Thanks,
Rajeshbabu.
---------- Forwarded message ----------
From: Dharmesh Guna <dg...@vmware.com>
Date: Thu, Jul 14, 2016 at 2:57 PM
Subject: RE: Issue in Bulkload when Table name contains '-' (hyphen)
To: "rajeshbabu@apache.org" <ch...@gmail.com>
Hi Rajesh,
Here is a select query.
CREATE TABLE "RB-TEST" (ID INTEGER PRIMARY KEY, NAME VARCHAR);
-Dharmesh
*From:* rajeshbabu@apache.org [mailto:chrajeshbabu32@gmail.com]
*Sent:* Thursday, July 14, 2016 2:45 PM
*To:* Dharmesh Guna
*Subject:* Re: Issue in Bulkload when Table name contains '-' (hyphen)
I see. Can you just provide the create table query?
Thanks,
Rajeshbabu.
On Thu, Jul 14, 2016 at 2:42 PM, Dharmesh Guna <dg...@vmware.com> wrote:
Thanks Rajesh for quick response.
Yes. I am able to upsert values from Phoenix sqlline thin client if I
mention table name as "DMG-TEST".
I will raise Jira ticket shortly.
-Dharmesh
-----Original Message-----
From: rajeshbabu@apache.org [mailto:chrajeshbabu32@gmail.com]
Sent: Thursday, July 14, 2016 2:39 PM
To: dev@phoenix.apache.org
Cc: dev-owner@phoenix.apache.org
Subject: Re: Issue in Bulkload when Table name contains '-' (hyphen)
Hi Dharmesh,
Are you able to write to table through normal upsert query? Seems like a
bug. You can raise a JIRA for this.
Thanks,
Rajeshbabu.
On Thu, Jul 14, 2016 at 11:22 AM, Dharmesh Guna <dg...@vmware.com> wrote:
> Dear All,
>
> I am facing issue in bulk loading from csv files into Phoenix when my
> Phoenix table contains '-' (hyphen) in table name.
> I am using Phoenix version 4.7.0. Phoenix supports '-' (hyphen) in
> table name as per naming conventions. [a-zA-Z_0-9-.] If I try to bulk
> load using below command it fails and gives below error.
>
> sudo -u hadoop
> HADOOP_CLASSPATH=/usr/lib/hbase/hbase-protocol.jar:/usr/lib/hbase/conf
> / hadoop jar /usr/lib/phoenix/phoenix-client.jar
> org.apache.phoenix.mapreduce.CsvBulkLoadTool
> -Dfs.permissions.umask-mode=000 -t 'DMG-TEST' --input
> "/user/test/1/DMG-TEST.csv" -d $'\t'
>
> So ultimately it removes any single or double quotes around table name
> before upserting which causes error 601.
>
> 2016-07-14 05:39:07,380 WARN [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics
> system already initialized!
> 2016-07-14 05:39:08,944 INFO [main]
> org.apache.phoenix.util.UpsertExecutor: Upserting SQL data with UPSERT
> INTO DMG-TEST ("APPLICATIONID", "0"."PACKAGEID") VALUES (?, ?)
> 2016-07-14 05:39:08,945 INFO [main] org.apache.hadoop.mapred.MapTask:
> Starting flush of map output
> 2016-07-14 05:39:08,952 INFO [main]
> org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor
> [.snappy]
> 2016-07-14 05:39:08,959 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException:
> org.apache.phoenix.exception.PhoenixParserException: ERROR 601 (42P00):
> Syntax error. Encountered "-" at line 1, column 17.
> at
>
org.apache.phoenix.util.UpsertExecutor.createStatement(UpsertExecutor.java:83)
> at
> org.apache.phoenix.util.UpsertExecutor.<init>(UpsertExecutor.java:94)
> at
>
org.apache.phoenix.util.csv.CsvUpsertExecutor.<init>(CsvUpsertExecutor.java:63)
> at
>
org.apache.phoenix.mapreduce.CsvToKeyValueMapper.buildUpsertExecutor(CsvToKeyValueMapper.java:85)
> at
>
org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.setup(FormatToBytesWritableMapper.java:142)
> at
>
org.apache.phoenix.mapreduce.CsvToKeyValueMapper.setup(CsvToKeyValueMapper.java:67)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:796)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
>
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.phoenix.exception.PhoenixParserException: ERROR
> 601
> (42P00): Syntax error. Encountered "-" at line 1, column 17.
> at
>
org.apache.phoenix.exception.PhoenixParserException.newException(PhoenixParserException.java:33)
> at
> org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:111)
> at
>
org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1185)
> at
>
org.apache.phoenix.jdbc.PhoenixStatement.parseStatement(PhoenixStatement.java:1268)
> at
>
org.apache.phoenix.jdbc.PhoenixPreparedStatement.<init>(PhoenixPreparedStatement.java:94)
> at
>
org.apache.phoenix.jdbc.PhoenixConnection.prepareStatement(PhoenixConnection.java:715)
> at
>
org.apache.phoenix.util.UpsertExecutor.createStatement(UpsertExecutor.java:81)
> ... 13 more
> Caused by: NoViableAltException(94@[])
> at
>
org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:4723)
> at
>
org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:775)
> at
>
org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:500)
> at
> org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> ... 18 more
>
>
> Thank you!!
>
> Regards,
> :: Dharmesh Guna
> :: Development | Custom Reports
> :: Contact | 961.199.87.52 | Ext. | 130 3264
>
>
RE: Issue in Bulkload when Table name contains '-' (hyphen)
Posted by Dharmesh Guna <dg...@vmware.com>.
Thanks Rajesh for quick response.
Yes. I am able to upsert values from Phoenix sqlline thin client if I mention table name as "DMG-TEST".
I will raise Jira ticket shortly.
-Dharmesh
-----Original Message-----
From: rajeshbabu@apache.org [mailto:chrajeshbabu32@gmail.com]
Sent: Thursday, July 14, 2016 2:39 PM
To: dev@phoenix.apache.org
Cc: dev-owner@phoenix.apache.org
Subject: Re: Issue in Bulkload when Table name contains '-' (hyphen)
Hi Dharmesh,
Are you able to write to table through normal upsert query? Seems like a bug. You can raise a JIRA for this.
Thanks,
Rajeshbabu.
On Thu, Jul 14, 2016 at 11:22 AM, Dharmesh Guna <dg...@vmware.com> wrote:
> Dear All,
>
> I am facing issue in bulk loading from csv files into Phoenix when my
> Phoenix table contains '-' (hyphen) in table name.
> I am using Phoenix version 4.7.0. Phoenix supports '-' (hyphen) in
> table name as per naming conventions. [a-zA-Z_0-9-.] If I try to bulk
> load using below command it fails and gives below error.
>
> sudo -u hadoop
> HADOOP_CLASSPATH=/usr/lib/hbase/hbase-protocol.jar:/usr/lib/hbase/conf
> / hadoop jar /usr/lib/phoenix/phoenix-client.jar
> org.apache.phoenix.mapreduce.CsvBulkLoadTool
> -Dfs.permissions.umask-mode=000 -t 'DMG-TEST' --input
> "/user/test/1/DMG-TEST.csv" -d $'\t'
>
> So ultimately it removes any single or double quotes around table name
> before upserting which causes error 601.
>
> 2016-07-14 05:39:07,380 WARN [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics
> system already initialized!
> 2016-07-14 05:39:08,944 INFO [main]
> org.apache.phoenix.util.UpsertExecutor: Upserting SQL data with UPSERT
> INTO DMG-TEST ("APPLICATIONID", "0"."PACKAGEID") VALUES (?, ?)
> 2016-07-14 05:39:08,945 INFO [main] org.apache.hadoop.mapred.MapTask:
> Starting flush of map output
> 2016-07-14 05:39:08,952 INFO [main]
> org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor
> [.snappy]
> 2016-07-14 05:39:08,959 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException:
> org.apache.phoenix.exception.PhoenixParserException: ERROR 601 (42P00):
> Syntax error. Encountered "-" at line 1, column 17.
> at
> org.apache.phoenix.util.UpsertExecutor.createStatement(UpsertExecutor.java:83)
> at
> org.apache.phoenix.util.UpsertExecutor.<init>(UpsertExecutor.java:94)
> at
> org.apache.phoenix.util.csv.CsvUpsertExecutor.<init>(CsvUpsertExecutor.java:63)
> at
> org.apache.phoenix.mapreduce.CsvToKeyValueMapper.buildUpsertExecutor(CsvToKeyValueMapper.java:85)
> at
> org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.setup(FormatToBytesWritableMapper.java:142)
> at
> org.apache.phoenix.mapreduce.CsvToKeyValueMapper.setup(CsvToKeyValueMapper.java:67)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:796)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.phoenix.exception.PhoenixParserException: ERROR
> 601
> (42P00): Syntax error. Encountered "-" at line 1, column 17.
> at
> org.apache.phoenix.exception.PhoenixParserException.newException(PhoenixParserException.java:33)
> at
> org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:111)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1185)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.parseStatement(PhoenixStatement.java:1268)
> at
> org.apache.phoenix.jdbc.PhoenixPreparedStatement.<init>(PhoenixPreparedStatement.java:94)
> at
> org.apache.phoenix.jdbc.PhoenixConnection.prepareStatement(PhoenixConnection.java:715)
> at
> org.apache.phoenix.util.UpsertExecutor.createStatement(UpsertExecutor.java:81)
> ... 13 more
> Caused by: NoViableAltException(94@[])
> at
> org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:4723)
> at
> org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:775)
> at
> org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:500)
> at
> org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> ... 18 more
>
>
> Thank you!!
>
> Regards,
> :: Dharmesh Guna
> :: Development | Custom Reports
> :: Contact | 961.199.87.52 | Ext. | 130 3264
>
>
Re: Issue in Bulkload when Table name contains '-' (hyphen)
Posted by "rajeshbabu@apache.org" <ch...@gmail.com>.
Hi Dharmesh,
Are you able to write to table through normal upsert query? Seems like a
bug. You can raise a JIRA for this.
Thanks,
Rajeshbabu.
On Thu, Jul 14, 2016 at 11:22 AM, Dharmesh Guna <dg...@vmware.com> wrote:
> Dear All,
>
> I am facing issue in bulk loading from csv files into Phoenix when my
> Phoenix table contains '-' (hyphen) in table name.
> I am using Phoenix version 4.7.0. Phoenix supports '-' (hyphen) in table
> name as per naming conventions. [a-zA-Z_0-9-.]
> If I try to bulk load using below command it fails and gives below error.
>
> sudo -u hadoop
> HADOOP_CLASSPATH=/usr/lib/hbase/hbase-protocol.jar:/usr/lib/hbase/conf/
> hadoop jar /usr/lib/phoenix/phoenix-client.jar
> org.apache.phoenix.mapreduce.CsvBulkLoadTool
> -Dfs.permissions.umask-mode=000 -t 'DMG-TEST' --input
> "/user/test/1/DMG-TEST.csv" -d $'\t'
>
> So ultimately it removes any single or double quotes around table name
> before upserting which causes error 601.
>
> 2016-07-14 05:39:07,380 WARN [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> already initialized!
> 2016-07-14 05:39:08,944 INFO [main]
> org.apache.phoenix.util.UpsertExecutor: Upserting SQL data with UPSERT
> INTO DMG-TEST ("APPLICATIONID", "0"."PACKAGEID") VALUES (?, ?)
> 2016-07-14 05:39:08,945 INFO [main] org.apache.hadoop.mapred.MapTask:
> Starting flush of map output
> 2016-07-14 05:39:08,952 INFO [main]
> org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.snappy]
> 2016-07-14 05:39:08,959 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException:
> org.apache.phoenix.exception.PhoenixParserException: ERROR 601 (42P00):
> Syntax error. Encountered "-" at line 1, column 17.
> at
> org.apache.phoenix.util.UpsertExecutor.createStatement(UpsertExecutor.java:83)
> at
> org.apache.phoenix.util.UpsertExecutor.<init>(UpsertExecutor.java:94)
> at
> org.apache.phoenix.util.csv.CsvUpsertExecutor.<init>(CsvUpsertExecutor.java:63)
> at
> org.apache.phoenix.mapreduce.CsvToKeyValueMapper.buildUpsertExecutor(CsvToKeyValueMapper.java:85)
> at
> org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.setup(FormatToBytesWritableMapper.java:142)
> at
> org.apache.phoenix.mapreduce.CsvToKeyValueMapper.setup(CsvToKeyValueMapper.java:67)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:796)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.phoenix.exception.PhoenixParserException: ERROR 601
> (42P00): Syntax error. Encountered "-" at line 1, column 17.
> at
> org.apache.phoenix.exception.PhoenixParserException.newException(PhoenixParserException.java:33)
> at
> org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:111)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1185)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.parseStatement(PhoenixStatement.java:1268)
> at
> org.apache.phoenix.jdbc.PhoenixPreparedStatement.<init>(PhoenixPreparedStatement.java:94)
> at
> org.apache.phoenix.jdbc.PhoenixConnection.prepareStatement(PhoenixConnection.java:715)
> at
> org.apache.phoenix.util.UpsertExecutor.createStatement(UpsertExecutor.java:81)
> ... 13 more
> Caused by: NoViableAltException(94@[])
> at
> org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:4723)
> at
> org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:775)
> at
> org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:500)
> at
> org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> ... 18 more
>
>
> Thank you!!
>
> Regards,
> :: Dharmesh Guna
> :: Development | Custom Reports
> :: Contact | 961.199.87.52 | Ext. | 130 3264
>
>