You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2019/08/16 00:51:21 UTC
[GitHub] [incubator-druid] zhangyihong opened a new issue #8321: why cause
my Druid cluster crashed during run indexing service
zhangyihong opened a new issue #8321: why cause my Druid cluster crashed during run indexing service
URL: https://github.com/apache/incubator-druid/issues/8321
these is middleManager logs :
```
2019-08-15T06:43:56,234 ERROR [forking-task-runner-68] com.google.common.util.concurrent.ExecutionList - RuntimeException while executing runnable com.google.common.util.concurrent.Futures$4@78cf894 with executor com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService@3fecd319
java.util.concurrent.RejectedExecutionException: Task org.apache.druid.indexing.worker.WorkerTaskManager$$Lambda$38/872018987@1339048e rejected from java.util.concurrent.ThreadPoolExecutor@301f24[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3029]
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) ~[?:1.8.0_211]
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) ~[?:1.8.0_211]
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) ~[?:1.8.0_211]
at java.util.concurrent.Executors$DelegatedExecutorService.execute(Executors.java:668) ~[?:1.8.0_211]
at org.apache.druid.indexing.worker.WorkerTaskManager.submitNoticeToExec(WorkerTaskManager.java:178) ~[druid-indexing-service-0.14.0-incubating-iap14.jar:0.14.0-incubating-iap14]
at org.apache.druid.indexing.worker.WorkerTaskManager.access$000(WorkerTaskManager.java:76) ~[druid-indexing-service-0.14.0-incubating-iap14.jar:0.14.0-incubating-iap14]
at org.apache.druid.indexing.worker.WorkerTaskManager$2.onSuccess(WorkerTaskManager.java:242) ~[druid-indexing-service-0.14.0-incubating-iap14.jar:0.14.0-incubating-iap14]
at org.apache.druid.indexing.worker.WorkerTaskManager$2.onSuccess(WorkerTaskManager.java:238) ~[druid-indexing-service-0.14.0-incubating-iap14.jar:0.14.0-incubating-iap14]
at com.google.common.util.concurrent.Futures$4.run(Futures.java:1181) ~[guava-16.0.1.jar:?]
at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) ~[guava-16.0.1.jar:?]
at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156) [guava-16.0.1.jar:?]
at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145) [guava-16.0.1.jar:?]
at com.google.common.util.concurrent.ListenableFutureTask.done(ListenableFutureTask.java:91) [guava-16.0.1.jar:?]
at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:384) [?:1.8.0_211]
at java.util.concurrent.FutureTask.set(FutureTask.java:233) [?:1.8.0_211]
at java.util.concurrent.FutureTask.run(FutureTask.java:274) [?:1.8.0_211]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_211]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_211]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_211]
2019-08-15T06:43:56,235 INFO [Thread-49] org.apache.druid.indexing.overlord.ForkingTaskRunner - Finished stopping in 14,028ms.
2019-08-15T06:43:56,235 INFO [Thread-49] org.apache.druid.indexing.worker.WorkerTaskManager - Stopped WorkerTaskManager.
2019-08-15T06:43:56,235 INFO [Thread-49] org.apache.druid.indexing.worker.WorkerTaskMonitor - Stopped WorkerTaskMonitor.
2019-08-15T06:43:56,235 INFO [Thread-49] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void org.apache.druid.discovery.DruidLeaderClient.stop()] on object[org.apache.druid.discovery.DruidLeaderClient@22f4f8f2].
2019-08-15T06:43:56,235 INFO [Thread-49] org.apache.druid.discovery.DruidLeaderClient - Stopped.
2019-08-15T06:43:56,235 INFO [Thread-49] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void org.apache.druid.curator.discovery.ServerDiscoverySelector.stop() throws java.io.IOException] on object[org.apache.druid.curator.discovery.ServerDiscoverySelector@1e1b061].
2019-08-15T06:43:56,236 INFO [Thread-49] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider.stop()] on object[org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider@3a790e40].
2019-08-15T06:43:56,236 INFO [Thread-49] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - stopping
2019-08-15T06:43:56,236 INFO [Thread-49] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - stopped
2019-08-15T06:43:56,236 INFO [Thread-49] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void org.apache.druid.java.util.http.client.NettyHttpClient.stop()] on object[org.apache.druid.java.util.http.client.NettyHttpClient@38830ea].
2019-08-15T06:43:56,246 INFO [Thread-49] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void org.apache.druid.indexing.worker.WorkerCuratorCoordinator.stop()] on object[org.apache.druid.indexing.worker.WorkerCuratorCoordinator@45cd7bc5].
2019-08-15T06:43:56,246 INFO [Thread-49] org.apache.druid.indexing.worker.WorkerCuratorCoordinator - Stopping WorkerCuratorCoordinator for worker[172.16.0.180:8091]
2019-08-15T06:43:56,246 INFO [Thread-49] org.apache.druid.curator.announcement.Announcer - Stopping announcer
2019-08-15T06:43:56,246 INFO [Thread-49] org.apache.druid.curator.announcement.Announcer - unannouncing [/druid/indexer/announcements/172.16.0.180:8091]
2019-08-15T06:43:56,247 INFO [Thread-49] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void org.apache.druid.java.util.metrics.MonitorScheduler.stop()] on object[org.apache.druid.java.util.metrics.MonitorScheduler@12bbfc54].
2019-08-15T06:43:56,248 INFO [Thread-49] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void org.apache.druid.java.util.emitter.service.ServiceEmitter.close() throws java.io.IOException] on object[ServiceEmitter{serviceDimensions={service=druid/middlemanager, host=172.16.0.180:8091, version=0.14.0-incubating-iap14}, emitter=LoggingEmitter{log=Logger{name=[org.apache.druid.java.util.emitter.core.LoggingEmitter], class[class org.apache.logging.slf4j.Log4jLogger]}, level=DEBUG}}].
2019-08-15T06:43:56,248 INFO [Thread-49] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void org.apache.druid.java.util.emitter.core.LoggingEmitter.close()] on object[LoggingEmitter{log=Logger{name=[org.apache.druid.java.util.emitter.core.LoggingEmitter], class[class org.apache.logging.slf4j.Log4jLogger]}, level=DEBUG}].
2019-08-15T06:43:56,248 INFO [Thread-49] org.apache.druid.curator.CuratorModule - Stopping Curator
2019-08-15T06:43:56,248 INFO [Curator-Framework-0] org.apache.curator.framework.imps.CuratorFrameworkImpl - backgroundOperationsLoop exiting
2019-08-15T06:43:56,249 INFO [Thread-49] org.apache.zookeeper.ZooKeeper - Session: 0x1000092abc10fd0 closed
2019-08-15T06:43:56,249 INFO [Thread-49] org.apache.druid.java.util.common.lifecycle.Lifecycle - Stopping lifecycle [module] stage [INIT]
2019-08-15T06:43:56,249 INFO [Thread-49] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void org.apache.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner.stop()] on object[org.apache.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner@2503ec73].
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org