You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Edward Capriolo (JIRA)" <ji...@apache.org> on 2008/10/30 21:34:44 UTC
[jira] Created: (HADOOP-4553) Create table not accepting COLLECTION
ITEM DELIMITER
Create table not accepting COLLECTION ITEM DELIMITER
----------------------------------------------------
Key: HADOOP-4553
URL: https://issues.apache.org/jira/browse/HADOOP-4553
Project: Hadoop Core
Issue Type: Bug
Components: contrib/hive
Affects Versions: 0.17.2
Environment: Using Hive dist from Facebook mirror, not very sure if this is the correct way to file this
Reporter: Edward Capriolo
COLLECTION ITEMS TERMINATED BY '\t'; always causes an error regardless of what '\t' '5' '\5' etc.
hive> create table testlist (id int,name string,favnum ARRAY<int>) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' COLLECTION ITEMS TERMINATED BY '\t';
java.lang.NumberFormatException: For input string: " "
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Integer.parseInt(Integer.java:447)
at java.lang.Byte.parseByte(Byte.java:151)
at java.lang.Byte.valueOf(Byte.java:184)
at java.lang.Byte.valueOf(Byte.java:208)
at org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol.initialize(TCTLSeparatedProtocol.java:316)
at org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe.initialize(DynamicSerDe.java:85)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:178)
at org.apache.hadoop.hive.ql.metadata.Table.initSerDe(Table.java:135)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:198)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:205)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:174)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:208)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:304)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
FAILED: Error in metadata: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException java.lang.NumberFormatException: For input string: " ")
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Time taken: 2.823 seconds
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4553) Create table not accepting
COLLECTION ITEM DELIMITER
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644185#action_12644185 ]
Zheng Shao commented on HADOOP-4553:
------------------------------------
After some discussions, I think we'd better drop that ascii code thing.
We will work on supporting '\t'.
> Create table not accepting COLLECTION ITEM DELIMITER
> ----------------------------------------------------
>
> Key: HADOOP-4553
> URL: https://issues.apache.org/jira/browse/HADOOP-4553
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hive
> Affects Versions: 0.17.2
> Environment: Using Hive dist from Facebook mirror, not very sure if this is the correct way to file this
> Reporter: Edward Capriolo
>
> COLLECTION ITEMS TERMINATED BY '\t'; always causes an error regardless of what '\t' '5' '\5' etc.
>
> hive> create table testlist (id int,name string,favnum ARRAY<int>) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' COLLECTION ITEMS TERMINATED BY '\t';
> java.lang.NumberFormatException: For input string: " "
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:447)
> at java.lang.Byte.parseByte(Byte.java:151)
> at java.lang.Byte.valueOf(Byte.java:184)
> at java.lang.Byte.valueOf(Byte.java:208)
> at org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol.initialize(TCTLSeparatedProtocol.java:316)
> at org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe.initialize(DynamicSerDe.java:85)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:178)
> at org.apache.hadoop.hive.ql.metadata.Table.initSerDe(Table.java:135)
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:198)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:205)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:174)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:208)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:304)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
> at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
> FAILED: Error in metadata: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException java.lang.NumberFormatException: For input string: " ")
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
> Time taken: 2.823 seconds
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4553) Create table not accepting
COLLECTION ITEM DELIMITER
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644130#action_12644130 ]
Zheng Shao commented on HADOOP-4553:
------------------------------------
Hi Edward, please use the ascii code of the separator for now ( 9 for tab, 32 for space):
create table testlist (id int,name string,favnum ARRAY<int>) ROW FORMAT DELIMITED FIELDS TERMINATED BY '9' COLLECTION ITEMS TERMINATED BY '32';
We will support both the ascii code and the string in the future. (That means we won't be able to use the string '9' as separator but I guess nobody would do that.)
> Create table not accepting COLLECTION ITEM DELIMITER
> ----------------------------------------------------
>
> Key: HADOOP-4553
> URL: https://issues.apache.org/jira/browse/HADOOP-4553
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hive
> Affects Versions: 0.17.2
> Environment: Using Hive dist from Facebook mirror, not very sure if this is the correct way to file this
> Reporter: Edward Capriolo
>
> COLLECTION ITEMS TERMINATED BY '\t'; always causes an error regardless of what '\t' '5' '\5' etc.
>
> hive> create table testlist (id int,name string,favnum ARRAY<int>) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' COLLECTION ITEMS TERMINATED BY '\t';
> java.lang.NumberFormatException: For input string: " "
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:447)
> at java.lang.Byte.parseByte(Byte.java:151)
> at java.lang.Byte.valueOf(Byte.java:184)
> at java.lang.Byte.valueOf(Byte.java:208)
> at org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol.initialize(TCTLSeparatedProtocol.java:316)
> at org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe.initialize(DynamicSerDe.java:85)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:178)
> at org.apache.hadoop.hive.ql.metadata.Table.initSerDe(Table.java:135)
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:198)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:205)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:174)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:208)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:304)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
> at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
> FAILED: Error in metadata: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException java.lang.NumberFormatException: For input string: " ")
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
> Time taken: 2.823 seconds
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4553) Create table not accepting
COLLECTION ITEM DELIMITER
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644198#action_12644198 ]
Edward Capriolo commented on HADOOP-4553:
-----------------------------------------
For an query string
a=b&c=d&e=f
It would be helpful to be able to use = or &
Are you saying only special escape characters will be supported or ascii will still be supported using the /0001 syntax?
> Create table not accepting COLLECTION ITEM DELIMITER
> ----------------------------------------------------
>
> Key: HADOOP-4553
> URL: https://issues.apache.org/jira/browse/HADOOP-4553
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hive
> Affects Versions: 0.17.2
> Environment: Using Hive dist from Facebook mirror, not very sure if this is the correct way to file this
> Reporter: Edward Capriolo
>
> COLLECTION ITEMS TERMINATED BY '\t'; always causes an error regardless of what '\t' '5' '\5' etc.
>
> hive> create table testlist (id int,name string,favnum ARRAY<int>) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' COLLECTION ITEMS TERMINATED BY '\t';
> java.lang.NumberFormatException: For input string: " "
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:447)
> at java.lang.Byte.parseByte(Byte.java:151)
> at java.lang.Byte.valueOf(Byte.java:184)
> at java.lang.Byte.valueOf(Byte.java:208)
> at org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol.initialize(TCTLSeparatedProtocol.java:316)
> at org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe.initialize(DynamicSerDe.java:85)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:178)
> at org.apache.hadoop.hive.ql.metadata.Table.initSerDe(Table.java:135)
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:198)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:205)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:174)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:208)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:304)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
> at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
> FAILED: Error in metadata: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException java.lang.NumberFormatException: For input string: " ")
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
> Time taken: 2.823 seconds
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4553) Create table not accepting
COLLECTION ITEM DELIMITER
Posted by "Prasad Chakka (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644203#action_12644203 ]
Prasad Chakka commented on HADOOP-4553:
---------------------------------------
one of the issues is that java bean's xml is 1.0 and we haven't found a way to store ^A like characters in xml 1.0 if there is it would be very trivial to support it.
> Create table not accepting COLLECTION ITEM DELIMITER
> ----------------------------------------------------
>
> Key: HADOOP-4553
> URL: https://issues.apache.org/jira/browse/HADOOP-4553
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hive
> Affects Versions: 0.17.2
> Environment: Using Hive dist from Facebook mirror, not very sure if this is the correct way to file this
> Reporter: Edward Capriolo
>
> COLLECTION ITEMS TERMINATED BY '\t'; always causes an error regardless of what '\t' '5' '\5' etc.
>
> hive> create table testlist (id int,name string,favnum ARRAY<int>) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' COLLECTION ITEMS TERMINATED BY '\t';
> java.lang.NumberFormatException: For input string: " "
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:447)
> at java.lang.Byte.parseByte(Byte.java:151)
> at java.lang.Byte.valueOf(Byte.java:184)
> at java.lang.Byte.valueOf(Byte.java:208)
> at org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol.initialize(TCTLSeparatedProtocol.java:316)
> at org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe.initialize(DynamicSerDe.java:85)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:178)
> at org.apache.hadoop.hive.ql.metadata.Table.initSerDe(Table.java:135)
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:198)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:205)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:174)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:208)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:304)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
> at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
> FAILED: Error in metadata: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException java.lang.NumberFormatException: For input string: " ")
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
> Time taken: 2.823 seconds
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.