You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Carl Steinbach (JIRA)" <ji...@apache.org> on 2010/12/28 04:56:45 UTC
[jira] Created: (HIVE-1868) ALTER TABLE ADD PARTITION can't handle
partition key names that conflict with HQL keywords
ALTER TABLE ADD PARTITION can't handle partition key names that conflict with HQL keywords
------------------------------------------------------------------------------------------
Key: HIVE-1868
URL: https://issues.apache.org/jira/browse/HIVE-1868
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Carl Steinbach
{code}
hive> CREATE TABLE foo (a INT) PARTITIONED BY (`date` STRING);
OK
hive> DESCRIBE EXTENDED foo;
OK
a int
date string
Detailed Table Information
Table(tableName:foo, dbName:default, owner:carl, createTime:1293508187,
lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:
[FieldSchema(name:a, type:int, comment:null),
FieldSchema(name:date, type:string, comment:null)], location:file:/user/hive/warehouse/foo,
inputFormat:org.apache.hadoop.mapred.TextInputFormat,
outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat,
compressed:false, numBuckets:-1,
serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe,
parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}),
partitionKeys:[FieldSchema(name:date, type:string, comment:null)],
parameters:{transient_lastDdlTime=1293508187}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE)
Time taken: 0.107 seconds
hive> ALTER TABLE foo ADD PARTITION (date='Feb 25 1980');
FAILED: Parse Error: line 1:31 mismatched input 'date' expecting Identifier in add partition statement
10/12/27 19:50:13 ERROR ql.Driver: FAILED: Parse Error: line 1:31 mismatched input 'date' expecting Identifier in add partition statement
org.apache.hadoop.hive.ql.parse.ParseException: line 1:31 mismatched input 'date' expecting Identifier in add partition statement
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:406)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:320)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:235)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:450)
hive> ALTER TABLE foo ADD PARTITION (`date`='Feb 25 1980');
ALTER TABLE foo ADD PARTITION (`date`='Feb 25 1980');
10/12/27 19:50:22 INFO parse.ParseDriver: Parsing command: ALTER TABLE foo ADD PARTITION (`date`='Feb 25 1980')
10/12/27 19:50:22 INFO parse.ParseDriver: Parse Completed
10/12/27 19:50:22 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=foo
10/12/27 19:50:22 INFO hive.log: DDL: struct foo { i32 a}
10/12/27 19:50:22 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=foo
10/12/27 19:50:22 INFO hive.log: DDL: struct foo { i32 a}
10/12/27 19:50:22 INFO ql.Driver: Semantic Analysis Completed
10/12/27 19:50:22 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null)
10/12/27 19:50:22 INFO ql.Driver: Starting command: ALTER TABLE foo ADD PARTITION (`date`='Feb 25 1980')
10/12/27 19:50:22 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=foo
10/12/27 19:50:22 INFO hive.log: DDL: struct foo { i32 a}
FAILED: Error in metadata: date not found in table's partition spec: {`date`=Feb 25 1980}
10/12/27 19:50:22 ERROR exec.DDLTask: FAILED: Error in metadata: date not found in table's partition spec: {`date`=Feb 25 1980}
org.apache.hadoop.hive.ql.metadata.HiveException: date not found in table's partition spec: {`date`=Feb 25 1980}
at org.apache.hadoop.hive.ql.metadata.Table.isValidSpec(Table.java:348)
at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1165)
at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1147)
at org.apache.hadoop.hive.ql.exec.DDLTask.addPartition(DDLTask.java:395)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:230)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:988)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:825)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:698)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:235)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:450)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
10/12/27 19:50:22 ERROR ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (HIVE-1868) ALTER TABLE ADD PARTITION can't
handle partition key names that conflict with HQL keywords
Posted by "Tamir Duberstein (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13490901#comment-13490901 ]
Tamir Duberstein commented on HIVE-1868:
----------------------------------------
2 years - what's going on with this?
> ALTER TABLE ADD PARTITION can't handle partition key names that conflict with HQL keywords
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-1868
> URL: https://issues.apache.org/jira/browse/HIVE-1868
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Carl Steinbach
>
> {code}
> hive> CREATE TABLE foo (a INT) PARTITIONED BY (`date` STRING);
> OK
> hive> DESCRIBE EXTENDED foo;
> OK
> a int
> date string
>
> Detailed Table Information
> Table(tableName:foo, dbName:default, owner:carl, createTime:1293508187,
> lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:
> [FieldSchema(name:a, type:int, comment:null),
> FieldSchema(name:date, type:string, comment:null)], location:file:/user/hive/warehouse/foo,
> inputFormat:org.apache.hadoop.mapred.TextInputFormat,
> outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat,
> compressed:false, numBuckets:-1,
> serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe,
> parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}),
> partitionKeys:[FieldSchema(name:date, type:string, comment:null)],
> parameters:{transient_lastDdlTime=1293508187}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE)
> Time taken: 0.107 seconds
> hive> ALTER TABLE foo ADD PARTITION (date='Feb 25 1980');
> FAILED: Parse Error: line 1:31 mismatched input 'date' expecting Identifier in add partition statement
> 10/12/27 19:50:13 ERROR ql.Driver: FAILED: Parse Error: line 1:31 mismatched input 'date' expecting Identifier in add partition statement
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:31 mismatched input 'date' expecting Identifier in add partition statement
> at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:406)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:320)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:235)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:450)
> hive> ALTER TABLE foo ADD PARTITION (`date`='Feb 25 1980');
> ALTER TABLE foo ADD PARTITION (`date`='Feb 25 1980');
> 10/12/27 19:50:22 INFO parse.ParseDriver: Parsing command: ALTER TABLE foo ADD PARTITION (`date`='Feb 25 1980')
> 10/12/27 19:50:22 INFO parse.ParseDriver: Parse Completed
> 10/12/27 19:50:22 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=foo
> 10/12/27 19:50:22 INFO hive.log: DDL: struct foo { i32 a}
> 10/12/27 19:50:22 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=foo
> 10/12/27 19:50:22 INFO hive.log: DDL: struct foo { i32 a}
> 10/12/27 19:50:22 INFO ql.Driver: Semantic Analysis Completed
> 10/12/27 19:50:22 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> 10/12/27 19:50:22 INFO ql.Driver: Starting command: ALTER TABLE foo ADD PARTITION (`date`='Feb 25 1980')
> 10/12/27 19:50:22 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=foo
> 10/12/27 19:50:22 INFO hive.log: DDL: struct foo { i32 a}
> FAILED: Error in metadata: date not found in table's partition spec: {`date`=Feb 25 1980}
> 10/12/27 19:50:22 ERROR exec.DDLTask: FAILED: Error in metadata: date not found in table's partition spec: {`date`=Feb 25 1980}
> org.apache.hadoop.hive.ql.metadata.HiveException: date not found in table's partition spec: {`date`=Feb 25 1980}
> at org.apache.hadoop.hive.ql.metadata.Table.isValidSpec(Table.java:348)
> at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1165)
> at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1147)
> at org.apache.hadoop.hive.ql.exec.DDLTask.addPartition(DDLTask.java:395)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:230)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:988)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:825)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:698)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:235)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:450)
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
> 10/12/27 19:50:22 ERROR ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
> hive>
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1868) ALTER TABLE ADD PARTITION can't
handle partition key names that conflict with HQL keywords
Posted by "Micah Caldwell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444646#comment-13444646 ]
Micah Caldwell commented on HIVE-1868:
--------------------------------------
This problem is not limited to keywords. The error appears to be that you can't use `` marks in ALTER TABLE ... ADD PARTITION (`...`='...'); This is easily worked around in any non-keyword case but unexpected behavior since column names are normally quoted with `.
> ALTER TABLE ADD PARTITION can't handle partition key names that conflict with HQL keywords
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-1868
> URL: https://issues.apache.org/jira/browse/HIVE-1868
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Carl Steinbach
>
> {code}
> hive> CREATE TABLE foo (a INT) PARTITIONED BY (`date` STRING);
> OK
> hive> DESCRIBE EXTENDED foo;
> OK
> a int
> date string
>
> Detailed Table Information
> Table(tableName:foo, dbName:default, owner:carl, createTime:1293508187,
> lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:
> [FieldSchema(name:a, type:int, comment:null),
> FieldSchema(name:date, type:string, comment:null)], location:file:/user/hive/warehouse/foo,
> inputFormat:org.apache.hadoop.mapred.TextInputFormat,
> outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat,
> compressed:false, numBuckets:-1,
> serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe,
> parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}),
> partitionKeys:[FieldSchema(name:date, type:string, comment:null)],
> parameters:{transient_lastDdlTime=1293508187}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE)
> Time taken: 0.107 seconds
> hive> ALTER TABLE foo ADD PARTITION (date='Feb 25 1980');
> FAILED: Parse Error: line 1:31 mismatched input 'date' expecting Identifier in add partition statement
> 10/12/27 19:50:13 ERROR ql.Driver: FAILED: Parse Error: line 1:31 mismatched input 'date' expecting Identifier in add partition statement
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:31 mismatched input 'date' expecting Identifier in add partition statement
> at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:406)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:320)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:235)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:450)
> hive> ALTER TABLE foo ADD PARTITION (`date`='Feb 25 1980');
> ALTER TABLE foo ADD PARTITION (`date`='Feb 25 1980');
> 10/12/27 19:50:22 INFO parse.ParseDriver: Parsing command: ALTER TABLE foo ADD PARTITION (`date`='Feb 25 1980')
> 10/12/27 19:50:22 INFO parse.ParseDriver: Parse Completed
> 10/12/27 19:50:22 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=foo
> 10/12/27 19:50:22 INFO hive.log: DDL: struct foo { i32 a}
> 10/12/27 19:50:22 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=foo
> 10/12/27 19:50:22 INFO hive.log: DDL: struct foo { i32 a}
> 10/12/27 19:50:22 INFO ql.Driver: Semantic Analysis Completed
> 10/12/27 19:50:22 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> 10/12/27 19:50:22 INFO ql.Driver: Starting command: ALTER TABLE foo ADD PARTITION (`date`='Feb 25 1980')
> 10/12/27 19:50:22 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=foo
> 10/12/27 19:50:22 INFO hive.log: DDL: struct foo { i32 a}
> FAILED: Error in metadata: date not found in table's partition spec: {`date`=Feb 25 1980}
> 10/12/27 19:50:22 ERROR exec.DDLTask: FAILED: Error in metadata: date not found in table's partition spec: {`date`=Feb 25 1980}
> org.apache.hadoop.hive.ql.metadata.HiveException: date not found in table's partition spec: {`date`=Feb 25 1980}
> at org.apache.hadoop.hive.ql.metadata.Table.isValidSpec(Table.java:348)
> at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1165)
> at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1147)
> at org.apache.hadoop.hive.ql.exec.DDLTask.addPartition(DDLTask.java:395)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:230)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:988)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:825)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:698)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:235)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:450)
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
> 10/12/27 19:50:22 ERROR ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
> hive>
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira