You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Ashish Kumar (Jira)" <ji...@apache.org> on 2021/04/04 08:34:00 UTC
[jira] [Comment Edited] (HIVE-23710) Add table meta cache limit
when starting Hive server2
[ https://issues.apache.org/jira/browse/HIVE-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314440#comment-17314440 ]
Ashish Kumar edited comment on HIVE-23710 at 4/4/21, 8:33 AM:
--------------------------------------------------------------
The given patch is not working for version 2.3.4
Modifying this file. -
metastore/src/java/org/apache/hadoop/hive/metastore/RetryingMetaStoreClient.java
Getting NullPointerException -
{quote}2021-04-04T13:37:26,016 INFO [main] server.HiveServer2: Starting HiveServer2
2021-04-04T13:37:26,237 INFO [main] service.AbstractService: Service:OperationManager is inited.
2021-04-04T13:37:26,237 INFO [main] service.AbstractService: Service:SessionManager is inited.
2021-04-04T13:37:26,237 INFO [main] service.AbstractService: Service:CLIService is inited.
2021-04-04T13:37:26,240 INFO [main] service.AbstractService: Service:ThriftBinaryCLIService is inited.
2021-04-04T13:37:26,240 INFO [main] service.AbstractService: Service:HiveServer2 is inited.
2021-04-04T13:37:26,241 INFO [main] server.HiveServer2: Shutting down HiveServer2
2021-04-04T13:37:26,242 WARN [main] server.HiveServer2: Error starting HiveServer2 on attempt 2, will retry in 60000ms
java.lang.NullPointerException
at com.sun.proxy.$Proxy36.isCompatibleWith(Unknown Source) ~[?:?]
at org.apache.hadoop.hive.ql.metadata.Hive.isCompatible(Hive.java:345) ~[hive-exec-2.3.4.jar:2.3.4]
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:311) ~[hive-exec-2.3.4.jar:2.3.4]
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:288) ~[hive-exec-2.3.4.jar:2.3.4]
at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:166) ~[hive-service-2.3.4.jar:2.3.4]
at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:607) [hive-service-2.3.4.jar:2.3.4]
at org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:100) [hive-service-2.3.4.jar:2.3.4]
at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:855) [hive-service-2.3.4.jar:2.3.4]
at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:724) [hive-service-2.3.4.jar:2.3.4]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_282]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_282]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_282]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_282]
at org.apache.hadoop.util.RunJar.run(RunJar.java:221) [hadoop-common-2.7.3.jar:?]
at org.apache.hadoop.util.RunJar.main(RunJar.java:136) [hadoop-common-2.7.3.jar:?]
{quote}
was (Author: ashubhumca):
The given patch is not working for version 2.4
Modifying this file. -
metastore/src/java/org/apache/hadoop/hive/metastore/RetryingMetaStoreClient.java
Getting NullPointerException -
{quote}2021-04-04T13:37:26,016 INFO [main] server.HiveServer2: Starting HiveServer2
2021-04-04T13:37:26,237 INFO [main] service.AbstractService: Service:OperationManager is inited.
2021-04-04T13:37:26,237 INFO [main] service.AbstractService: Service:SessionManager is inited.
2021-04-04T13:37:26,237 INFO [main] service.AbstractService: Service:CLIService is inited.
2021-04-04T13:37:26,240 INFO [main] service.AbstractService: Service:ThriftBinaryCLIService is inited.
2021-04-04T13:37:26,240 INFO [main] service.AbstractService: Service:HiveServer2 is inited.
2021-04-04T13:37:26,241 INFO [main] server.HiveServer2: Shutting down HiveServer2
2021-04-04T13:37:26,242 WARN [main] server.HiveServer2: Error starting HiveServer2 on attempt 2, will retry in 60000ms
java.lang.NullPointerException
at com.sun.proxy.$Proxy36.isCompatibleWith(Unknown Source) ~[?:?]
at org.apache.hadoop.hive.ql.metadata.Hive.isCompatible(Hive.java:345) ~[hive-exec-2.3.4.jar:2.3.4]
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:311) ~[hive-exec-2.3.4.jar:2.3.4]
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:288) ~[hive-exec-2.3.4.jar:2.3.4]
at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:166) ~[hive-service-2.3.4.jar:2.3.4]
at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:607) [hive-service-2.3.4.jar:2.3.4]
at org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:100) [hive-service-2.3.4.jar:2.3.4]
at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:855) [hive-service-2.3.4.jar:2.3.4]
at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:724) [hive-service-2.3.4.jar:2.3.4]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_282]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_282]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_282]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_282]
at org.apache.hadoop.util.RunJar.run(RunJar.java:221) [hadoop-common-2.7.3.jar:?]
at org.apache.hadoop.util.RunJar.main(RunJar.java:136) [hadoop-common-2.7.3.jar:?]
{quote}
> Add table meta cache limit when starting Hive server2
> -----------------------------------------------------
>
> Key: HIVE-23710
> URL: https://issues.apache.org/jira/browse/HIVE-23710
> Project: Hive
> Issue Type: Improvement
> Environment: Hive 2.3.6
> Reporter: Deegue
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23710.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When we start up Hive server2, it will connect to metastore to get table meta info by database and cache them. If there are many tables in a database, however, will exceed `hive.metastore.client.socket.timeout`.
> Then exception thrown like:
> {noformat}
> 2020-06-17T11:38:27,595 WARN [main] metastore.RetryingMetaStoreClient: MetaStoreClient lost connection. Attempting to reconnect (1 of 1) after 1s. getTableObjectsByName
> org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129) ~[hive-exec-2.3.6.jar:2.3.6]
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) ~[hive-exec-2.3.6.jar:2.3.6]
> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) ~[hive-exec-2.3.6.jar:2.3.6]
> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) ~[hive-exec-2.3.6.jar:2.3.6]
> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) ~[hive-exec-2.3.6.jar:2.3.6]
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77) ~[hive-exec-2.3.6.jar:2.3.6]
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table_objects_by_name_req(ThriftHiveMetastore.java:1596) ~[hive-exec-2.3.6.jar:2.3.6]
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table_objects_by_name_req(ThriftHiveMetastore.java:1583) ~[hive-exec-2.3.6.jar:2.3.6]
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTableObjectsByName(HiveMetaStoreClient.java:1370) ~[hive-exec-2.3.6.jar:2.3.6]
> at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.getTableObjectsByName(SessionHiveMetaStoreClient.java:238) ~[hive-exec-2.3.6.jar:2.3.6]
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) ~[?:?]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:206) ~[hive-exec-2.3.6.jar:2.3.6]
> at com.sun.proxy.$Proxy38.getTableObjectsByName(Unknown Source) ~[?:?]
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) ~[?:?]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2336) ~[hive-exec-2.3.6.jar:2.3.6]
> at com.sun.proxy.$Proxy38.getTableObjectsByName(Unknown Source) ~[?:?]
> at org.apache.hadoop.hive.ql.metadata.Hive.getAllTableObjects(Hive.java:1343) ~[hive-exec-2.3.6.jar:2.3.6]
> at org.apache.hadoop.hive.ql.metadata.HiveMaterializedViewsRegistry.init(HiveMaterializedViewsRegistry.java:127) ~[hive-exec-2.3.6.jar:2.3.6]
> at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:167) ~[hive-service-2.3.6.jar:2.3.6]
> at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:607) ~[hive-service-2.3.6.jar:2.3.6]
> at org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:100) ~[hive-service-2.3.6.jar:2.3.6]
> at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:855) ~[hive-service-2.3.6.jar:2.3.6]
> at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:724) ~[hive-service-2.3.6.jar:2.3.6]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
> at org.apache.hadoop.util.RunJar.run(RunJar.java:226) ~[hadoop-common-2.6.0-cdh5.16.1.jar:?]
> at org.apache.hadoop.util.RunJar.main(RunJar.java:141) ~[hadoop-common-2.6.0-cdh5.16.1.jar:?]
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_121]
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_121]
> at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_121]
> at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_121]
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[?:1.8.0_121]
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) ~[?:1.8.0_121]
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345) ~[?:1.8.0_121]
> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127) ~[hive-exec-2.3.6.jar:2.3.6]
> ... 32 more
> {noformat}
> The solution is add `hive.server2.init.load.table.limit` to control max number of tables for one TCP request from Hive server2 to metastore.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)