You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Daniel Dai (Created) (JIRA)" <ji...@apache.org> on 2012/02/11 02:18:59 UTC
[jira] [Created] (HIVE-2800) NPE in "create index" without comment
clause in external metastore
NPE in "create index" without comment clause in external metastore
------------------------------------------------------------------
Key: HIVE-2800
URL: https://issues.apache.org/jira/browse/HIVE-2800
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.8.1, 0.9.0
Reporter: Daniel Dai
Priority: Minor
This happens only when using external metastore (with --hiveconf hive.metastore.uris=thrift://localhost:8088 --hiveconf hive.metastore.local=false). Also if I gave a comment in the statement, this exception go away.
Here is the statement:
create index test111 on table hcat_test(name) as 'compact' with deferred rebuild;
Here is the stack:
2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - FAILED: Error in metadata: java.lang.NullPointerException
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
at org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.NullPointerException
at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
... 17 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2800) NPE in "create index" without comment
clause in external metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan reassigned HIVE-2800:
--------------------------------------
Assignee: Sushanth Sowmyan
> NPE in "create index" without comment clause in external metastore
> ------------------------------------------------------------------
>
> Key: HIVE-2800
> URL: https://issues.apache.org/jira/browse/HIVE-2800
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.1, 0.8.1, 0.9.0
> Reporter: Daniel Dai
> Assignee: Sushanth Sowmyan
> Priority: Minor
>
> This happens only when using external metastore (with --hiveconf hive.metastore.uris=thrift://localhost:8088 --hiveconf hive.metastore.local=false). Also if I gave a comment in the statement, this exception go away.
> Here is the statement:
> create index test111 on table hcat_test(name) as 'compact' with deferred rebuild;
> Here is the stack:
> 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - FAILED: Error in metadata: java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NullPointerException
> at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
> at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
> ... 17 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2800) NPE in "create index" without
comment clause in external metastore
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291385#comment-13291385 ]
Sushanth Sowmyan commented on HIVE-2800:
----------------------------------------
This thread makes for interesting reading, and refers to the same issue:
http://www.mail-archive.com/thrift-dev@incubator.apache.org/msg01185.html
> NPE in "create index" without comment clause in external metastore
> ------------------------------------------------------------------
>
> Key: HIVE-2800
> URL: https://issues.apache.org/jira/browse/HIVE-2800
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.1, 0.8.1, 0.9.0
> Reporter: Daniel Dai
> Assignee: Sushanth Sowmyan
> Priority: Minor
>
> This happens only when using external metastore (with --hiveconf hive.metastore.uris=thrift://localhost:8088 --hiveconf hive.metastore.local=false). Also if I gave a comment in the statement, this exception go away.
> Here is the statement:
> create index test111 on table hcat_test(name) as 'compact' with deferred rebuild;
> Here is the stack:
> 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - FAILED: Error in metadata: java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NullPointerException
> at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
> at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
> ... 17 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2800) NPE in "create index" without comment
clause in external metastore
Posted by "Bejoy KS (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bejoy KS updated HIVE-2800:
---------------------------
Affects Version/s: 0.7.1
> NPE in "create index" without comment clause in external metastore
> ------------------------------------------------------------------
>
> Key: HIVE-2800
> URL: https://issues.apache.org/jira/browse/HIVE-2800
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.1, 0.8.1, 0.9.0
> Reporter: Daniel Dai
> Priority: Minor
>
> This happens only when using external metastore (with --hiveconf hive.metastore.uris=thrift://localhost:8088 --hiveconf hive.metastore.local=false). Also if I gave a comment in the statement, this exception go away.
> Here is the statement:
> create index test111 on table hcat_test(name) as 'compact' with deferred rebuild;
> Here is the stack:
> 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - FAILED: Error in metadata: java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NullPointerException
> at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
> at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
> ... 17 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2800) NPE in "create index" without
comment clause in external metastore
Posted by "Bejoy KS (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280054#comment-13280054 ]
Bejoy KS commented on HIVE-2800:
--------------------------------
The same bug is found on hive 0.7.1 as well
> NPE in "create index" without comment clause in external metastore
> ------------------------------------------------------------------
>
> Key: HIVE-2800
> URL: https://issues.apache.org/jira/browse/HIVE-2800
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.1, 0.8.1, 0.9.0
> Reporter: Daniel Dai
> Priority: Minor
>
> This happens only when using external metastore (with --hiveconf hive.metastore.uris=thrift://localhost:8088 --hiveconf hive.metastore.local=false). Also if I gave a comment in the statement, this exception go away.
> Here is the statement:
> create index test111 on table hcat_test(name) as 'compact' with deferred rebuild;
> Here is the stack:
> 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - FAILED: Error in metadata: java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NullPointerException
> at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
> at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
> ... 17 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2800) NPE in "create index" without
comment clause in external metastore
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291347#comment-13291347 ]
Sushanth Sowmyan commented on HIVE-2800:
----------------------------------------
Hmm, actually, the problem does go deeper than that. We'd have the same issue if a user were to set one of the parameter values as null. We'd fix this issue by this patch, but only to hit it again with other user code.
> NPE in "create index" without comment clause in external metastore
> ------------------------------------------------------------------
>
> Key: HIVE-2800
> URL: https://issues.apache.org/jira/browse/HIVE-2800
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.1, 0.8.1, 0.9.0
> Reporter: Daniel Dai
> Assignee: Sushanth Sowmyan
> Priority: Minor
>
> This happens only when using external metastore (with --hiveconf hive.metastore.uris=thrift://localhost:8088 --hiveconf hive.metastore.local=false). Also if I gave a comment in the statement, this exception go away.
> Here is the statement:
> create index test111 on table hcat_test(name) as 'compact' with deferred rebuild;
> Here is the stack:
> 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - FAILED: Error in metadata: java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NullPointerException
> at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
> at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
> ... 17 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2800) NPE in "create index" without
comment clause in external metastore
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506120#comment-13506120 ]
Phabricator commented on HIVE-2800:
-----------------------------------
khorgath has abandoned the revision "HIVE-2800 [jira] NPE in "create index" without comment clause in external metastore".
Same change implemented by HIVE-3722, redundant now.
REVISION DETAIL
https://reviews.facebook.net/D3519
To: JIRA, ashutoshc, khorgath
> NPE in "create index" without comment clause in external metastore
> ------------------------------------------------------------------
>
> Key: HIVE-2800
> URL: https://issues.apache.org/jira/browse/HIVE-2800
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.1, 0.8.1, 0.9.0
> Reporter: Daniel Dai
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.10.0
>
>
> This happens only when using external metastore (with --hiveconf hive.metastore.uris=thrift://localhost:8088 --hiveconf hive.metastore.local=false). Also if I gave a comment in the statement, this exception go away.
> Here is the statement:
> create index test111 on table hcat_test(name) as 'compact' with deferred rebuild;
> Here is the stack:
> 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - FAILED: Error in metadata: java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NullPointerException
> at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
> at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
> ... 17 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2800) NPE in "create index" without
comment clause in external metastore
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290575#comment-13290575 ]
Sushanth Sowmyan commented on HIVE-2800:
----------------------------------------
Sample code that demonstrates bug:
{code}
drop table if exists missing_ddl_3;
drop index if exists missing_ddl_3_index on missing_ddl_3;
create table missing_ddl_3(name string, age int, gpa double);
insert overwrite table missing_ddl_3 select name, age, gpa from studenttab10k;
create index missing_ddl_3_index on table missing_ddl_3(name) as 'COMPACT' WITH DEFERRED REBUILD;
alter index missing_ddl_3_index on missing_ddl_3 rebuild;
{code}
This will fail at the create index time itself(ignoring the rebuild).
> NPE in "create index" without comment clause in external metastore
> ------------------------------------------------------------------
>
> Key: HIVE-2800
> URL: https://issues.apache.org/jira/browse/HIVE-2800
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.1, 0.8.1, 0.9.0
> Reporter: Daniel Dai
> Priority: Minor
>
> This happens only when using external metastore (with --hiveconf hive.metastore.uris=thrift://localhost:8088 --hiveconf hive.metastore.local=false). Also if I gave a comment in the statement, this exception go away.
> Here is the statement:
> create index test111 on table hcat_test(name) as 'compact' with deferred rebuild;
> Here is the stack:
> 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - FAILED: Error in metadata: java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NullPointerException
> at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
> at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
> ... 17 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2800) NPE in "create index" without
comment clause in external metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291088#comment-13291088 ]
Ashutosh Chauhan commented on HIVE-2800:
----------------------------------------
@Sushanth,
Reason it happens only in thrift client-server mode and your suggested fix of null-checking strongly hints to me that thrift layer is not able to handle nulls very well. This is kind of surprise to me as I thought thrift can handle nulls. I think there is a value in figuring out if this really is the case and file a bug with thrift project since it looks like this may bite us again in future.
> NPE in "create index" without comment clause in external metastore
> ------------------------------------------------------------------
>
> Key: HIVE-2800
> URL: https://issues.apache.org/jira/browse/HIVE-2800
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.1, 0.8.1, 0.9.0
> Reporter: Daniel Dai
> Priority: Minor
>
> This happens only when using external metastore (with --hiveconf hive.metastore.uris=thrift://localhost:8088 --hiveconf hive.metastore.local=false). Also if I gave a comment in the statement, this exception go away.
> Here is the statement:
> create index test111 on table hcat_test(name) as 'compact' with deferred rebuild;
> Here is the stack:
> 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - FAILED: Error in metadata: java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NullPointerException
> at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
> at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
> ... 17 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2800) NPE in "create index" without comment
clause in external metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan resolved HIVE-2800.
------------------------------------
Resolution: Fixed
Fix Version/s: 0.10.0
Fixed as a part of HIVE-3722
> NPE in "create index" without comment clause in external metastore
> ------------------------------------------------------------------
>
> Key: HIVE-2800
> URL: https://issues.apache.org/jira/browse/HIVE-2800
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.1, 0.8.1, 0.9.0
> Reporter: Daniel Dai
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.10.0
>
>
> This happens only when using external metastore (with --hiveconf hive.metastore.uris=thrift://localhost:8088 --hiveconf hive.metastore.local=false). Also if I gave a comment in the statement, this exception go away.
> Here is the statement:
> create index test111 on table hcat_test(name) as 'compact' with deferred rebuild;
> Here is the stack:
> 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - FAILED: Error in metadata: java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NullPointerException
> at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
> at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
> ... 17 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2800) NPE in "create index" without
comment clause in external metastore
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290537#comment-13290537 ]
Sushanth Sowmyan commented on HIVE-2800:
----------------------------------------
This seems to be because the we don't check if the comment is null before we add it. Then, later, when we're trying to write out using thrift, thrift throws the NPE when it tries to read from parameters and tries to write out as-is. The following fixes this error:
--- ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java (revision 1347150)
+++ ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java (working copy)
@@ -709,7 +709,10 @@
Index indexDesc = new Index(indexName, indexHandlerClass, dbName, tableName, time, time, indexTblName,
storageDescriptor, params, deferredRebuild);
- indexDesc.getParameters().put("comment", indexComment);
+ if (indexComment != null)
+ {
+ indexDesc.getParameters().put("comment", indexComment);
+ }
if (idxProps != null)
{
> NPE in "create index" without comment clause in external metastore
> ------------------------------------------------------------------
>
> Key: HIVE-2800
> URL: https://issues.apache.org/jira/browse/HIVE-2800
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.1, 0.8.1, 0.9.0
> Reporter: Daniel Dai
> Priority: Minor
>
> This happens only when using external metastore (with --hiveconf hive.metastore.uris=thrift://localhost:8088 --hiveconf hive.metastore.local=false). Also if I gave a comment in the statement, this exception go away.
> Here is the statement:
> create index test111 on table hcat_test(name) as 'compact' with deferred rebuild;
> Here is the stack:
> 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - FAILED: Error in metadata: java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NullPointerException
> at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
> at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
> ... 17 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2800) NPE in "create index" without
comment clause in external metastore
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291334#comment-13291334 ]
Sushanth Sowmyan commented on HIVE-2800:
----------------------------------------
Ashutosh, digging a bit more into thrift sources, this is what I find:
http://svn.apache.org/repos/asf/thrift/trunk/lib/java/src/org/apache/thrift/protocol/TBinaryProtocol.java
If you look at the method writeString there, it does not check for null, it calls str.getBytes(). So as of thrift trunk, this does not work. I'm not sure if that's intentional on thrift's part (to maximize speed by minimizing checks) and something to fix on hive's side, or something to fix on thrift's side.
In the meanwhile, I've added a testcase for this bug in the RemoteHiveMetaStore, I'm updating the patch now.
> NPE in "create index" without comment clause in external metastore
> ------------------------------------------------------------------
>
> Key: HIVE-2800
> URL: https://issues.apache.org/jira/browse/HIVE-2800
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.1, 0.8.1, 0.9.0
> Reporter: Daniel Dai
> Assignee: Sushanth Sowmyan
> Priority: Minor
>
> This happens only when using external metastore (with --hiveconf hive.metastore.uris=thrift://localhost:8088 --hiveconf hive.metastore.local=false). Also if I gave a comment in the statement, this exception go away.
> Here is the statement:
> create index test111 on table hcat_test(name) as 'compact' with deferred rebuild;
> Here is the stack:
> 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - FAILED: Error in metadata: java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NullPointerException
> at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
> at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
> ... 17 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2800) NPE in "create index" without
comment clause in external metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291355#comment-13291355 ]
Ashutosh Chauhan commented on HIVE-2800:
----------------------------------------
Thanks, Sushanth for digging in. I have created THRIFT-1625 to get response from thrift devs. Thats what my worry is if thrift can't handle nulls cleanly then we have to litter our codebase with null-checks everywhere.
> NPE in "create index" without comment clause in external metastore
> ------------------------------------------------------------------
>
> Key: HIVE-2800
> URL: https://issues.apache.org/jira/browse/HIVE-2800
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.1, 0.8.1, 0.9.0
> Reporter: Daniel Dai
> Assignee: Sushanth Sowmyan
> Priority: Minor
>
> This happens only when using external metastore (with --hiveconf hive.metastore.uris=thrift://localhost:8088 --hiveconf hive.metastore.local=false). Also if I gave a comment in the statement, this exception go away.
> Here is the statement:
> create index test111 on table hcat_test(name) as 'compact' with deferred rebuild;
> Here is the stack:
> 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - FAILED: Error in metadata: java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NullPointerException
> at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
> at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
> ... 17 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira