You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Kishore Yellamraju <ki...@rocketfuelinc.com> on 2013/04/16 21:26:27 UTC
Re: Using Python thrift client for Hive Server 2 hangs.
+1 .
+ user@hive.apache.org , may be this question is to hive users.
even we are having issues with running python scripts against hive. ( cdh4
, hive 0.10 , python 2.7 ) . these were working fine before and stopped
working after upgrade.
i see the below exception when running my python script.
Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.thrift.EncodingUtils.setBit(BIZ)B
at
org.apache.hadoop.hive.metastore.api.StorageDescriptor.setCompressedIsSet(StorageDescriptor.java:442)
at
org.apache.hadoop.hive.metastore.api.StorageDescriptor.<init>(StorageDescriptor.java:223)
at
org.apache.hadoop.hive.metastore.ObjectStore.convertToStorageDescriptor(ObjectStore.java:1002)
at
org.apache.hadoop.hive.metastore.ObjectStore.convertToStorageDescriptor(ObjectStore.java:1017)
at
org.apache.hadoop.hive.metastore.ObjectStore.convertToTable(ObjectStore.java:872)
at
org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:743)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:111)
at $Proxy9.getTable(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1340)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
at $Proxy10.get_table(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:780)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:74)
at $Proxy11.getTable(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:922)
at
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableAddParts(DDLSemanticAnalyzer.java:2269)
at
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:319)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:434)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:896)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:261)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:218)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:414)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:349)
-Thanks
kishore kumar yellamraju
On Tue, Apr 16, 2013 at 7:28 AM, Vidhya Venkataraman <vi...@dropbox.com>wrote:
> I have been trying to use the python client for server2 (I am using CDH4,
> Hive 0.10.x) and every execute function on the client hangs on
> recv_execute. I am using Python 2.7 for running the client.
>
> I am running the hive server on the same node as the client. I set the
> hive server logging level to console and I see no logged messages.
>
> Am I missing something obvious?
>
> V
>
> --
>
>
>
>
Re: Using Python thrift client for Hive Server 2 hangs.
Posted by Prasad Mujumdar <pr...@cloudera.com>.
HiveServer2 by default uses ThriftSASL transport. The Thrift python
library doesn't have the SASL transport. You can run the hiveserver2 in
non-sasl mode by adding following to the hive-site.xml
<property>
<name>hive.server2.authentication</name>
<value>NOSASL</value>
</property>
Also Hive 0.10 upgraded Thrift requirement from 0.7 to 0.9. You might want
to check the thrift jars in your classpath.
thanks
Prasad
On Tue, Apr 16, 2013 at 12:26 PM, Kishore Yellamraju <
kishore@rocketfuelinc.com> wrote:
> +1 .
>
> + user@hive.apache.org , may be this question is to hive users.
>
> even we are having issues with running python scripts against hive. ( cdh4
> , hive 0.10 , python 2.7 ) . these were working fine before and stopped
> working after upgrade.
>
> i see the below exception when running my python script.
>
>
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.apache.thrift.EncodingUtils.setBit(BIZ)B
> at
> org.apache.hadoop.hive.metastore.api.StorageDescriptor.setCompressedIsSet(StorageDescriptor.java:442)
> at
> org.apache.hadoop.hive.metastore.api.StorageDescriptor.<init>(StorageDescriptor.java:223)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.convertToStorageDescriptor(ObjectStore.java:1002)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.convertToStorageDescriptor(ObjectStore.java:1017)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.convertToTable(ObjectStore.java:872)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:743)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:111)
> at $Proxy9.getTable(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1340)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
> at $Proxy10.get_table(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:780)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:74)
> at $Proxy11.getTable(Unknown Source)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:922)
> at
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableAddParts(DDLSemanticAnalyzer.java:2269)
> at
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:319)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:434)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:896)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:261)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:218)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:414)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:349)
>
>
>
> -Thanks
> kishore kumar yellamraju
>
>
> On Tue, Apr 16, 2013 at 7:28 AM, Vidhya Venkataraman <vi...@dropbox.com>wrote:
>
>> I have been trying to use the python client for server2 (I am using CDH4,
>> Hive 0.10.x) and every execute function on the client hangs on
>> recv_execute. I am using Python 2.7 for running the client.
>>
>> I am running the hive server on the same node as the client. I set the
>> hive server logging level to console and I see no logged messages.
>>
>> Am I missing something obvious?
>>
>> V
>>
>> --
>>
>>
>>
>>
>
>
Re: Using Python thrift client for Hive Server 2 hangs.
Posted by Kishore Yellamraju <ki...@rocketfuelinc.com>.
Any thoughts on this issue of running python scripts against hive/cdh4.2 ?
-Thanks
kishore kumar yellamraju |Ground control
operations|kishore@rocketfuel.com| 408.203.042
4
On Tue, Apr 16, 2013 at 12:26 PM, Kishore Yellamraju <
kishore@rocketfuelinc.com> wrote:
> +1 .
>
> + user@hive.apache.org , may be this question is to hive users.
>
> even we are having issues with running python scripts against hive. ( cdh4
> , hive 0.10 , python 2.7 ) . these were working fine before and stopped
> working after upgrade.
>
> i see the below exception when running my python script.
>
>
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.apache.thrift.EncodingUtils.setBit(BIZ)B
> at
> org.apache.hadoop.hive.metastore.api.StorageDescriptor.setCompressedIsSet(StorageDescriptor.java:442)
> at
> org.apache.hadoop.hive.metastore.api.StorageDescriptor.<init>(StorageDescriptor.java:223)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.convertToStorageDescriptor(ObjectStore.java:1002)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.convertToStorageDescriptor(ObjectStore.java:1017)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.convertToTable(ObjectStore.java:872)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:743)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:111)
> at $Proxy9.getTable(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1340)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
> at $Proxy10.get_table(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:780)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:74)
> at $Proxy11.getTable(Unknown Source)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:922)
> at
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableAddParts(DDLSemanticAnalyzer.java:2269)
> at
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:319)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:434)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:896)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:261)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:218)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:414)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:349)
>
>
>
> -Thanks
> kishore kumar yellamraju
>
>
> On Tue, Apr 16, 2013 at 7:28 AM, Vidhya Venkataraman <vi...@dropbox.com>wrote:
>
>> I have been trying to use the python client for server2 (I am using CDH4,
>> Hive 0.10.x) and every execute function on the client hangs on
>> recv_execute. I am using Python 2.7 for running the client.
>>
>> I am running the hive server on the same node as the client. I set the
>> hive server logging level to console and I see no logged messages.
>>
>> Am I missing something obvious?
>>
>> V
>>
>> --
>>
>>
>>
>>
>
>