You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/04/06 14:29:43 UTC
[GitHub] [druid] tanisdlj commented on issue #10866: Ingestion task fails with InterruptedException when handling the segments
tanisdlj commented on issue #10866:
URL: https://github.com/apache/druid/issues/10866#issuecomment-814167420
Ok, another crash:
```
2021-04-06T13:36:36,486 INFO [qtp1718695429-151] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Stopping gracefully (status: [PUBLISHING])
2021-04-06T13:36:36,490 ERROR [task-runner-0-priority-0] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Encountered exception in run() before persisting.
org.apache.kafka.common.errors.InterruptException: java.lang.InterruptedException
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.maybeThrowInterruptException(ConsumerNetworkClient.java:520) ~[?:?]
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:281) ~[?:?]
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236) ~[?:?]
at org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1301) ~[?:?]
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1237) ~[?:?]
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1210) ~[?:?]
at org.apache.druid.indexing.kafka.KafkaRecordSupplier.poll(KafkaRecordSupplier.java:124) ~[?:?]
at org.apache.druid.indexing.kafka.IncrementalPublishingKafkaIndexTaskRunner.getRecords(IncrementalPublishingKafkaIndexTaskRunner.java:98) ~[?:?]
at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.runInternal(SeekableStreamIndexTaskRunner.java:603) [druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.run(SeekableStreamIndexTaskRunner.java:267) [druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTask.run(SeekableStreamIndexTask.java:145) [druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:451) [druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:423) [druid-indexing-service-0.20.1.jar:0.20.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
Caused by: java.lang.InterruptedException
... 17 more
2021-04-06T13:36:36,527 ERROR [task-runner-0-priority-0] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Error while publishing segments for sequenceNumber[SequenceMetadata{sequenceId=12, sequenceName='index_kafka_ssp-events-hourly_4e62ca9663a43dc_12', assignments=[], startOffsets={70=4049765636, 10=97308684185}, exclusiveStartPartitions=[], endOffsets={70=4051651832, 10=97310341952}, sentinel=false, checkpointed=true}]
java.util.concurrent.CancellationException: Task was cancelled.
at com.google.common.util.concurrent.AbstractFuture.cancellationExceptionWithCause(AbstractFuture.java:392) ~[guava-16.0.1.jar:?]
at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:306) ~[guava-16.0.1.jar:?]
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286) ~[guava-16.0.1.jar:?]
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116) ~[guava-16.0.1.jar:?]
at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:135) ~[guava-16.0.1.jar:?]
at com.google.common.util.concurrent.Futures$4.run(Futures.java:1170) [guava-16.0.1.jar:?]
at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) [guava-16.0.1.jar:?]
at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156) [guava-16.0.1.jar:?]
at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145) [guava-16.0.1.jar:?]
at com.google.common.util.concurrent.AbstractFuture.cancel(AbstractFuture.java:134) [guava-16.0.1.jar:?]
at com.google.common.util.concurrent.Futures$ChainingListenableFuture.cancel(Futures.java:826) [guava-16.0.1.jar:?]
at com.google.common.util.concurrent.Futures$CombinedFuture$1.run(Futures.java:1505) [guava-16.0.1.jar:?]
at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) [guava-16.0.1.jar:?]
at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156) [guava-16.0.1.jar:?]
at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145) [guava-16.0.1.jar:?]
at com.google.common.util.concurrent.AbstractFuture.cancel(AbstractFuture.java:134) [guava-16.0.1.jar:?]
at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.runInternal(SeekableStreamIndexTaskRunner.java:867) [druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.run(SeekableStreamIndexTaskRunner.java:267) [druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTask.run(SeekableStreamIndexTask.java:145) [druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:451) [druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:423) [druid-indexing-service-0.20.1.jar:0.20.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
Caused by: java.util.concurrent.CancellationException: Future.cancel() was called.
at com.google.common.util.concurrent.AbstractFuture$Sync.complete(AbstractFuture.java:378) ~[guava-16.0.1.jar:?]
at com.google.common.util.concurrent.AbstractFuture$Sync.cancel(AbstractFuture.java:355) ~[guava-16.0.1.jar:?]
at com.google.common.util.concurrent.AbstractFuture.cancel(AbstractFuture.java:131) ~[guava-16.0.1.jar:?]
... 15 more
2021-04-06T13:36:36,545 INFO [task-runner-0-priority-0] org.apache.druid.server.coordination.BatchDataSegmentAnnouncer - Unannouncing segment[ssp-events-hourly_2021-04-06T13:00:00.000Z_2021-04-06T14:00:00.000Z_2021-04-06T13:00:01.025Z_103] at path[/druid/segments/druid-dataserver-11:8100/druid-dataserver-11:8100_indexer-executor__default_tier_2021-04-06T09:24:09.621Z_11bf1d0b81054d63a80702952d9e13700]
2021-04-06T13:36:36,546 INFO [task-runner-0-priority-0] org.apache.druid.server.coordination.BatchDataSegmentAnnouncer - Unannouncing segment[ssp-events-hourly_2021-04-06T12:00:00.000Z_2021-04-06T13:00:00.000Z_2021-04-06T12:00:00.048Z_283] at path[/druid/segments/druid-dataserver-11:8100/druid-dataserver-11:8100_indexer-executor__default_tier_2021-04-06T09:24:09.621Z_11bf1d0b81054d63a80702952d9e13700]
2021-04-06T13:36:36,546 INFO [task-runner-0-priority-0] org.apache.druid.server.coordination.BatchDataSegmentAnnouncer - Unannouncing segment[ssp-events-hourly_2021-04-06T13:00:00.000Z_2021-04-06T14:00:00.000Z_2021-04-06T13:00:01.025Z_30] at path[/druid/segments/druid-dataserver-11:8100/druid-dataserver-:8100_indexer-executor__default_tier_2021-04-06T09:24:09.621Z_11bf1d0b81054d63a80702952d9e13700]
2021-04-06T13:36:36,546 INFO [task-runner-0-priority-0] org.apache.druid.curator.announcement.Announcer - Unannouncing [/druid/segments/druid-dataserver-11:8100/druid-dataserver-11:8100_indexer-executor__default_tier_2021-04-06T09:24:09.621Z_11bf1d0b81054d63a80702952d9e13700]
2021-04-06T13:36:37,104 WARN [[index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc]-appenderator-merge] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Failed to push merged index for segment[ssp-events-hourly_2021-04-06T12:00:00.000Z_2021-04-06T13:00:00.000Z_2021-04-06T12:00:00.048Z_283].
java.nio.channels.ClosedByInterruptException: null
at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) ~[?:1.8.0_275]
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:216) ~[?:1.8.0_275]
at org.apache.druid.io.Channels.writeFully(Channels.java:32) ~[druid-core-0.20.1.jar:0.20.1]
at org.apache.druid.segment.writeout.FileWriteOutBytes.flush(FileWriteOutBytes.java:62) ~[druid-processing-0.20.1.jar:0.20.1]
at org.apache.druid.segment.writeout.FileWriteOutBytes.flushIfNeeded(FileWriteOutBytes.java:54) ~[druid-processing-0.20.1.jar:0.20.1]
at org.apache.druid.segment.writeout.FileWriteOutBytes.write(FileWriteOutBytes.java:86) ~[druid-processing-0.20.1.jar:0.20.1]
at org.apache.druid.segment.data.GenericIndexedWriter$1.writeTo(GenericIndexedWriter.java:123) ~[druid-processing-0.20.1.jar:0.20.1]
at org.apache.druid.segment.data.GenericIndexedWriter$1.writeTo(GenericIndexedWriter.java:96) ~[druid-processing-0.20.1.jar:0.20.1]
at org.apache.druid.segment.data.GenericIndexedWriter.write(GenericIndexedWriter.java:237) ~[druid-processing-0.20.1.jar:0.20.1]
at org.apache.druid.segment.data.BlockLayoutColumnarDoublesSerializer.add(BlockLayoutColumnarDoublesSerializer.java:95) ~[druid-processing-0.20.1.jar:0.20.1]
at org.apache.druid.segment.DoubleColumnSerializer.serialize(DoubleColumnSerializer.java:83) ~[druid-processing-0.20.1.jar:0.20.1]
at org.apache.druid.segment.IndexMergerV9.mergeIndexesAndWriteColumns(IndexMergerV9.java:573) ~[druid-processing-0.20.1.jar:0.20.1]
at org.apache.druid.segment.IndexMergerV9.makeIndexFiles(IndexMergerV9.java:267) ~[druid-processing-0.20.1.jar:0.20.1]
at org.apache.druid.segment.IndexMergerV9.merge(IndexMergerV9.java:999) ~[druid-processing-0.20.1.jar:0.20.1]
at org.apache.druid.segment.IndexMergerV9.mergeQueryableIndex(IndexMergerV9.java:917) ~[druid-processing-0.20.1.jar:0.20.1]
at org.apache.druid.segment.IndexMergerV9.mergeQueryableIndex(IndexMergerV9.java:895) ~[druid-processing-0.20.1.jar:0.20.1]
at org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.mergeAndPush(AppenderatorImpl.java:769) ~[druid-server-0.20.1.jar:0.20.1]
at org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$push$1(AppenderatorImpl.java:660) ~[druid-server-0.20.1.jar:0.20.1]
at com.google.common.util.concurrent.Futures$1.apply(Futures.java:713) [guava-16.0.1.jar:?]
at com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:861) [guava-16.0.1.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
2021-04-06T13:36:37,133 ERROR [[index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Incremental persist failed: {class=org.apache.druid.segment.realtime.appenderator.AppenderatorImpl, segment=ssp-events-hourly_2021-04-06T13:00:00.000Z_2021-04-06T14:00:00.000Z_2021-04-06T13:00:01.025Z_103, dataSource=ssp-events-hourly, count=9}
2021-04-06T13:36:37,157 INFO [task-runner-0-priority-0] org.apache.druid.curator.announcement.Announcer - Unannouncing [/druid/internal-discovery/PEON/druid-dataserver-11:8100]
2021-04-06T13:36:37,164 INFO [task-runner-0-priority-0] org.apache.druid.curator.discovery.CuratorDruidNodeAnnouncer - Unannounced self [{"druidNode":{"service":"druid/middleManager","host":"druid-dataserver-11","bindOnHost":false,"plaintextPort":8100,"port":-1,"tlsPort":-1,"enablePlaintextPort":true,"enableTlsPort":false},"nodeType":"peon","services":{"dataNodeService":{"type":"dataNodeService","tier":"_default_tier","maxSize":0,"type":"indexer-executor","priority":0},"lookupNodeService":{"type":"lookupNodeService","lookupTier":"__default"}}}].
2021-04-06T13:36:37,165 INFO [task-runner-0-priority-0] org.apache.druid.curator.announcement.Announcer - Unannouncing [/druid/announcements/druid-dataserver-11:8100]
2021-04-06T13:36:37,179 ERROR [task-runner-0-priority-0] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Encountered exception while running task.
org.apache.kafka.common.errors.InterruptException: java.lang.InterruptedException
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.maybeThrowInterruptException(ConsumerNetworkClient.java:520) ~[?:?]
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:281) ~[?:?]
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236) ~[?:?]
at org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1301) ~[?:?]
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1237) ~[?:?]
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1210) ~[?:?]
at org.apache.druid.indexing.kafka.KafkaRecordSupplier.poll(KafkaRecordSupplier.java:124) ~[?:?]
at org.apache.druid.indexing.kafka.IncrementalPublishingKafkaIndexTaskRunner.getRecords(IncrementalPublishingKafkaIndexTaskRunner.java:98) ~[?:?]
at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.runInternal(SeekableStreamIndexTaskRunner.java:603) ~[druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.run(SeekableStreamIndexTaskRunner.java:267) [druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTask.run(SeekableStreamIndexTask.java:145) [druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:451) [druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:423) [druid-indexing-service-0.20.1.jar:0.20.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
Suppressed: java.lang.InterruptedException
at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_275]
at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:1.8.0_275]
at org.apache.druid.segment.realtime.appenderator.StreamAppenderatorDriver.persist(StreamAppenderatorDriver.java:231) ~[druid-server-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.runInternal(SeekableStreamIndexTaskRunner.java:756) ~[druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.run(SeekableStreamIndexTaskRunner.java:267) [druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTask.run(SeekableStreamIndexTask.java:145) [druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:451) [druid-indexing-service-0.20.1.jar:0.20.1]
at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:423) [druid-indexing-service-0.20.1.jar:0.20.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
Caused by: java.lang.InterruptedException
... 17 more
2021-04-06T13:36:37,217 INFO [task-runner-0-priority-0] org.apache.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: {
"id" : "index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc",
"status" : "FAILED",
"duration" : 15153268,
"errorMsg" : "org.apache.kafka.common.errors.InterruptException: java.lang.InterruptedException\n\tat org.apache.kaf...",
"location" : {
"host" : null,
"port" : -1,
"tlsPort" : -1
}
}
2021-04-06T13:36:37,229 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Stopping lifecycle [module] stage [ANNOUNCEMENTS]
2021-04-06T13:36:37,230 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Stopping lifecycle [module] stage [SERVER]
2021-04-06T13:37:07,254 INFO [main] org.eclipse.jetty.server.AbstractConnector - Stopped ServerConnector@73dbe25{HTTP/1.1, (http/1.1)}{0.0.0.0:8100}
2021-04-06T13:37:07,255 INFO [main] org.eclipse.jetty.server.session - node0 Stopped scavenging
2021-04-06T13:37:07,259 INFO [main] org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.s.ServletContextHandler@6d35ff7b{/,null,STOPPED}
2021-04-06T13:37:07,266 ERROR [main] org.apache.druid.server.initialization.jetty.JettyServerModule - Jetty lifecycle event failed [class org.eclipse.jetty.server.Server]
java.util.concurrent.TimeoutException: null
at org.eclipse.jetty.util.FutureCallback.get(FutureCallback.java:130) ~[jetty-util-9.4.34.v20201102.jar:9.4.34.v20201102]
at org.eclipse.jetty.util.FutureCallback.get(FutureCallback.java:30) ~[jetty-util-9.4.34.v20201102.jar:9.4.34.v20201102]
at org.eclipse.jetty.server.handler.AbstractHandlerContainer.doShutdown(AbstractHandlerContainer.java:175) ~[jetty-server-9.4.34.v20201102.jar:9.4.34.v20201102]
at org.eclipse.jetty.server.Server.doStop(Server.java:447) ~[jetty-server-9.4.34.v20201102.jar:9.4.34.v20201102]
at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:94) [jetty-util-9.4.34.v20201102.jar:9.4.34.v20201102]
at org.apache.druid.server.initialization.jetty.JettyServerModule$2.stop(JettyServerModule.java:454) [druid-server-0.20.1.jar:0.20.1]
at org.apache.druid.java.util.common.lifecycle.Lifecycle.stop(Lifecycle.java:368) [druid-core-0.20.1.jar:0.20.1]
at org.apache.druid.cli.CliPeon.run(CliPeon.java:306) [druid-services-0.20.1.jar:0.20.1]
at org.apache.druid.cli.Main.main(Main.java:113) [druid-services-0.20.1.jar:0.20.1]
2021-04-06T13:37:07,270 WARN [main] org.apache.druid.server.initialization.jetty.JettyServerModule - Unable to stop Jetty server.
java.util.concurrent.TimeoutException: null
at org.eclipse.jetty.util.FutureCallback.get(FutureCallback.java:130) ~[jetty-util-9.4.34.v20201102.jar:9.4.34.v20201102]
at org.eclipse.jetty.util.FutureCallback.get(FutureCallback.java:30) ~[jetty-util-9.4.34.v20201102.jar:9.4.34.v20201102]
at org.eclipse.jetty.server.handler.AbstractHandlerContainer.doShutdown(AbstractHandlerContainer.java:175) ~[jetty-server-9.4.34.v20201102.jar:9.4.34.v20201102]
at org.eclipse.jetty.server.Server.doStop(Server.java:447) ~[jetty-server-9.4.34.v20201102.jar:9.4.34.v20201102]
at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:94) ~[jetty-util-9.4.34.v20201102.jar:9.4.34.v20201102]
at org.apache.druid.server.initialization.jetty.JettyServerModule$2.stop(JettyServerModule.java:454) [druid-server-0.20.1.jar:0.20.1]
at org.apache.druid.java.util.common.lifecycle.Lifecycle.stop(Lifecycle.java:368) [druid-core-0.20.1.jar:0.20.1]
at org.apache.druid.cli.CliPeon.run(CliPeon.java:306) [druid-services-0.20.1.jar:0.20.1]
at org.apache.druid.cli.Main.main(Main.java:113) [druid-services-0.20.1.jar:0.20.1]
2021-04-06T13:37:07,271 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Stopping lifecycle [module] stage [NORMAL]
2021-04-06T13:37:07,272 INFO [main] org.apache.druid.server.coordination.ZkCoordinator - Stopping ZkCoordinator for [DruidServerMetadata{name='druid-dataserver-11:8100', hostAndPort='druid-dataserver-1:8100', hostAndTlsPort='null', maxSize=0, tier='_default_tier', type=indexer-executor, priority=0}]
2021-04-06T13:37:07,272 INFO [main] org.apache.druid.server.coordination.SegmentLoadDropHandler - Stopping...
2021-04-06T13:37:07,272 INFO [main] org.apache.druid.server.coordination.SegmentLoadDropHandler - Stopped.
2021-04-06T13:37:07,277 INFO [main] org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner - Starting graceful shutdown of task[index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc].
2021-04-06T13:37:07,277 INFO [main] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Stopping gracefully (status: [PUBLISHING])
2021-04-06T13:37:07,285 INFO [main] org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner - Graceful shutdown of task[index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc] finished in 0ms.
2021-04-06T13:37:07,353 INFO [LookupExtractorFactoryContainerProvider-MainThread] org.apache.druid.query.lookup.LookupReferencesManager - Lookup Management loop exited. Lookup notices are not handled anymore.
2021-04-06T13:37:07,354 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/publishers.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='token', valueFieldName='IIN'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@59ffeff7
2021-04-06T13:37:07,356 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [publisherProductIIN].
2021-04-06T13:37:07,356 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/adtagConfigs.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='id', valueFieldName='creativePublicationId'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@3cc22ab4
2021-04-06T13:37:07,356 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [adtagCreativePublicationId].
2021-04-06T13:37:07,356 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/publishers.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='token', valueFieldName='tier'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@13bfbdc2
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [publisherTier].
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/adtagConfigs.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='id', valueFieldName='creativePublicationName'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@4d3fea62
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [adtagCreativePublicationName].
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/adtagConfigs.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='id', valueFieldName='videoDuration'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@2860a0af
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [adtagVideoDuration].
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/adtagConfigs.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='id', valueFieldName='adtagName'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@6a3ae078
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [adtagName].
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/adtagConfigs.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='id', valueFieldName='creativeId'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@4ff4c694
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [adtagCreativeId].
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/publishers.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='token', valueFieldName='IID'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@32a8e87a
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [publisherProductIID].
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/publishers.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='token', valueFieldName='FID'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@57380ab8
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [publisherProductFID].
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/deals.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='dealId', valueFieldName='name'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@31c4da0d
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [dealName].
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/publishers.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='token', valueFieldName='country'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@53078487
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [publisherCountry].
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/publishers.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='token', valueFieldName='id'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@3c6f005f
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [publisherId].
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/publishers.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='token', valueFieldName='editorialGroupName'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@64fd134f
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [editorialGroupName].
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/publishers.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='token', valueFieldName='PIV'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@16050083
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [publisherProductPIV].
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/publishers.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='token', valueFieldName='categoryName'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@3850830a
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [publisherCategory].
2021-04-06T13:37:07,357 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/publishers.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='token', valueFieldName='name'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@60c28b08
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [publisherName].
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/adtagConfigs.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='id', valueFieldName='productFamily'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@1107325c
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [adtagProductFamily].
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/adtagConfigs.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='id', valueFieldName='size'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@25c01974
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [adtagSize].
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/adtagConfigs.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='id', valueFieldName='extAdserver'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@43f81db5
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [adtagExtAdserver].
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/publishers.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='token', valueFieldName='IVD'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@30a3d5e3
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [publisherProductIVD].
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/adtagConfigs.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='id', valueFieldName='channel'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@79c0a348
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [adtagChannel].
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/publishers.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='token', valueFieldName='IIV'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@6021c46e
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [publisherProductIIV].
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/publishers.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='token', valueFieldName='FIV'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@47c78112
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [publisherProductFIV].
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=file:///usr/share/druid/lookups/publishers.json, uriPrefix=null, namespaceParseSpec=JSONFlatDataParser{keyFieldName='token', valueFieldName='IIT'}, fileRegex='null', pollPeriod=PT30M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@7d520a64
2021-04-06T13:37:07,358 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [publisherProductIIT].
2021-04-06T13:37:07,416 INFO [Curator-Framework-0] org.apache.curator.framework.imps.CuratorFrameworkImpl - backgroundOperationsLoop exiting
2021-04-06T13:37:07,422 INFO [main] org.apache.zookeeper.ZooKeeper - Session: 0x1034ca014c22804 closed
2021-04-06T13:37:07,422 INFO [main-EventThread] org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x1034ca014c22804
2021-04-06T13:37:07,422 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$CloseableHandler - Closing object[org.asynchttpclient.DefaultAsyncHttpClient@7be3abaa]
2021-04-06T13:37:07,434 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Stopping lifecycle [module] stage [INIT]
Finished peon task```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org