You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@sentry.apache.org by "Hao Hao (JIRA)" <ji...@apache.org> on 2017/02/15 20:39:41 UTC
[jira] [Updated] (SENTRY-1630) When HMSFollower exits in an
abnormal way, encounter out of sequence error for the following connection.
[ https://issues.apache.org/jira/browse/SENTRY-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hao Hao updated SENTRY-1630:
----------------------------
Description:
When HMSFollower exits in an abnormal way, for all the following connection would encounter out of sequence and SocketTimeoutException: Read timed out. Looking at HIVE-6893, it seems to relate to leakage connection problem.
{noformat}2017-02-15 19:03:42,822 ERROR org.apache.sentry.hdfs.FullUpdateInitializer: Task did not complete successfully after 0 tries. Exception got: org.apache.thrift.TApplicationException: get_database failed: out of sequence response
2017-02-15 19:03:42,827 ERROR org.apache.sentry.hdfs.FullUpdateInitializer: Task did not complete successfully after 0 tries. Exception got: MetaException(message:Got exception: org.apache.thrift.transport.TTransportException null)
2017-02-15 19:03:43,803 INFO hive.metastore: Closed a connection to metastore, current connections: 0
2017-02-15 19:03:43,803 ERROR org.apache.sentry.service.thrift.HMSFollower: Exception occurred persisting Hive full snapshot into DB
java.lang.RuntimeException: org.apache.thrift.TApplicationException: get_database failed: out of sequence response
at org.apache.sentry.hdfs.FullUpdateInitializer.createInitialUpdate(FullUpdateInitializer.java:324)
at org.apache.sentry.service.thrift.HMSFollower.fetchFullUpdate(HMSFollower.java:343)
at org.apache.sentry.service.thrift.HMSFollower.run(HMSFollower.java:244)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.TApplicationException: get_database failed: out of sequence response
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:84)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_database(ThriftHiveMetastore.java:662)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_database(ThriftHiveMetastore.java:649)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClient.java:1213)
at org.apache.sentry.hdfs.FullUpdateInitializer$DbTask.doTask(FullUpdateInitializer.java:256)
at org.apache.sentry.hdfs.FullUpdateInitializer$BaseTask$RetryStrategy.exec(FullUpdateInitializer.java:110)
at org.apache.sentry.hdfs.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:150)
at org.apache.sentry.hdfs.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
... 3 more
2017-02-15 19:03:43,849 INFO org.apache.sentry.service.thrift.HMSFollower: Making a kerberos connection to HMS{noformat}
was:When HMSFollower exits in an abnormal way, for all the following connection would encounter out of sequence and SocketTimeoutException: Read timed out. Looking at HIVE-6893, it seems to relate to leakage connection problem.
> When HMSFollower exits in an abnormal way, encounter out of sequence error for the following connection.
> --------------------------------------------------------------------------------------------------------
>
> Key: SENTRY-1630
> URL: https://issues.apache.org/jira/browse/SENTRY-1630
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
> Affects Versions: sentry-ha-redesign
> Reporter: Hao Hao
> Fix For: sentry-ha-redesign
>
>
> When HMSFollower exits in an abnormal way, for all the following connection would encounter out of sequence and SocketTimeoutException: Read timed out. Looking at HIVE-6893, it seems to relate to leakage connection problem.
> {noformat}2017-02-15 19:03:42,822 ERROR org.apache.sentry.hdfs.FullUpdateInitializer: Task did not complete successfully after 0 tries. Exception got: org.apache.thrift.TApplicationException: get_database failed: out of sequence response
> 2017-02-15 19:03:42,827 ERROR org.apache.sentry.hdfs.FullUpdateInitializer: Task did not complete successfully after 0 tries. Exception got: MetaException(message:Got exception: org.apache.thrift.transport.TTransportException null)
> 2017-02-15 19:03:43,803 INFO hive.metastore: Closed a connection to metastore, current connections: 0
> 2017-02-15 19:03:43,803 ERROR org.apache.sentry.service.thrift.HMSFollower: Exception occurred persisting Hive full snapshot into DB
> java.lang.RuntimeException: org.apache.thrift.TApplicationException: get_database failed: out of sequence response
> at org.apache.sentry.hdfs.FullUpdateInitializer.createInitialUpdate(FullUpdateInitializer.java:324)
> at org.apache.sentry.service.thrift.HMSFollower.fetchFullUpdate(HMSFollower.java:343)
> at org.apache.sentry.service.thrift.HMSFollower.run(HMSFollower.java:244)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.thrift.TApplicationException: get_database failed: out of sequence response
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:84)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_database(ThriftHiveMetastore.java:662)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_database(ThriftHiveMetastore.java:649)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClient.java:1213)
> at org.apache.sentry.hdfs.FullUpdateInitializer$DbTask.doTask(FullUpdateInitializer.java:256)
> at org.apache.sentry.hdfs.FullUpdateInitializer$BaseTask$RetryStrategy.exec(FullUpdateInitializer.java:110)
> at org.apache.sentry.hdfs.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:150)
> at org.apache.sentry.hdfs.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:79)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ... 3 more
> 2017-02-15 19:03:43,849 INFO org.apache.sentry.service.thrift.HMSFollower: Making a kerberos connection to HMS{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)