You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/01/05 17:00:33 UTC
[GitHub] [druid] mstein11 commented on issue #12123: Kafka Indexing does not fail ingestion task on OffsetOutOfRangeException even when no reset policy is configured
mstein11 commented on issue #12123:
URL: https://github.com/apache/druid/issues/12123#issuecomment-1005903411
Full Peon Log:
2022-01-05T11:50:15,791 INFO [main] org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator 5.2.5.Final
2022-01-05T11:50:16,272 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-histogram], jars: druid-histogram-0.22.1.jar
2022-01-05T11:50:16,273 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-datasketches], jars: commons-math3-3.6.1.jar, druid-datasketches-0.22.1.jar
2022-01-05T11:50:16,275 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-lookups-cached-global], jars: druid-lookups-cached-global-0.22.1.jar, mapdb-1.0.8.jar
2022-01-05T11:50:16,276 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [postgresql-metadata-storage], jars: postgresql-42.2.14.jar, postgresql-metadata-storage-0.22.1.jar
2022-01-05T11:50:16,276 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-kafka-indexing-service], jars: druid-kafka-indexing-service-0.22.1.jar, kafka-clients-2.8.0.jar, lz4-java-1.7.1.jar, snappy-java-1.1.8.1.jar, zstd-jni-1.3.3-1.jar
2022-01-05T11:50:16,278 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-basic-security], jars: druid-basic-security-0.22.1.jar
2022-01-05T11:50:16,435 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-histogram], jars: druid-histogram-0.22.1.jar
2022-01-05T11:50:16,437 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-datasketches], jars: commons-math3-3.6.1.jar, druid-datasketches-0.22.1.jar
2022-01-05T11:50:16,443 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-lookups-cached-global], jars: druid-lookups-cached-global-0.22.1.jar, mapdb-1.0.8.jar
2022-01-05T11:50:16,445 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [postgresql-metadata-storage], jars: postgresql-42.2.14.jar, postgresql-metadata-storage-0.22.1.jar
2022-01-05T11:50:16,446 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-kafka-indexing-service], jars: druid-kafka-indexing-service-0.22.1.jar, kafka-clients-2.8.0.jar, lz4-java-1.7.1.jar, snappy-java-1.1.8.1.jar, zstd-jni-1.3.3-1.jar
2022-01-05T11:50:16,448 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-basic-security], jars: druid-basic-security-0.22.1.jar
2022-01-05T11:50:17,594 INFO [main] org.apache.druid.guice.JsonConfigurator - Skipping druid.emitter.http.recipientBaseUrl property: one of it's prefixes is also used as a property key. Prefix: druid
2022-01-05T11:50:17,642 INFO [main] org.apache.druid.server.emitter.EmitterModule - Using emitter [HttpPostEmitter{config=HttpEmitterConfig{flushMillis=60000, flushCount=500, flushTimeOut=9223372036854775807, basicAuthentication='null', batchingStrategy=ARRAY, maxBatchSize=5242880, contentEncoding=null, batchQueueSizeLimit=6, httpTimeoutAllowanceFactor=2.0, minHttpTimeoutMillis=0, recipientBaseUrl='http://int-global-prometheus-druid-exporter:8080/druid'}}] for metrics and alerts, with dimensions [{version=0.22.1}].
2022-01-05T11:50:18,129 INFO [main] org.apache.druid.server.metrics.MetricsModule - Loaded 3 monitors: org.apache.druid.java.util.metrics.JvmMonitor, org.apache.druid.query.ExecutorServiceMonitor, org.apache.druid.server.initialization.jetty.JettyServerModule$JettyMonitor
2022-01-05T11:50:18,133 INFO [main] org.apache.druid.cli.CliPeon - Starting up with processors[2], memory[219,152,384], maxMemory[358,088,704], directMemory[201,326,592]. Properties follow.
2022-01-05T11:50:18,133 INFO [main] org.apache.druid.cli.CliPeon - * awt.toolkit: sun.awt.X11.XToolkit
2022-01-05T11:50:18,133 INFO [main] org.apache.druid.cli.CliPeon - * druid.auth.authenticator.MetadataAuthenticator.authorizerName: MetadataAuthorizer
2022-01-05T11:50:18,133 INFO [main] org.apache.druid.cli.CliPeon - * druid.auth.authenticator.MetadataAuthenticator.credentialsValidator.type: metadata
2022-01-05T11:50:18,133 INFO [main] org.apache.druid.cli.CliPeon - * druid.auth.authenticator.MetadataAuthenticator.initialAdminPassword: <masked>
2022-01-05T11:50:18,133 INFO [main] org.apache.druid.cli.CliPeon - * druid.auth.authenticator.MetadataAuthenticator.initialInternalClientPassword: <masked>
2022-01-05T11:50:18,133 INFO [main] org.apache.druid.cli.CliPeon - * druid.auth.authenticator.MetadataAuthenticator.skipOnFailure: false
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.auth.authenticator.MetadataAuthenticator.type: basic
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.auth.authenticatorChain: ["MetadataAuthenticator"]
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.auth.authorizer.MetadataAuthorizer.type: basic
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.auth.authorizers: ["MetadataAuthorizer"]
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.emitter: http
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.emitter.http.recipientBaseUrl: http://int-global-prometheus-druid-exporter:8080/druid
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.emitter.logging.logLevel: info
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.escalator.authorizerName: MetadataAuthorizer
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.escalator.internalClientPassword: <masked>
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.escalator.internalClientUsername: druid_system
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.escalator.type: basic
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.extensions.loadList: ["druid-histogram", "druid-datasketches", "druid-lookups-cached-global", "postgresql-metadata-storage", "druid-kafka-indexing-service", "druid-basic-security"]
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.generic.useDefaultValueForNull: false
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.global.http.readTimeout: PT1h
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.host: 10.85.6.250
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.indexer.fork.property.druid.processing.buffer.sizeBytes: 2500000
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.indexer.fork.property.druid.processing.numMergeBuffers: 2
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.indexer.fork.property.druid.processing.numThreads: 1
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.indexer.logs.directory: /opt/data/indexing-logs
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.indexer.logs.type: file
2022-01-05T11:50:18,134 INFO [main] org.apache.druid.cli.CliPeon - * druid.indexer.runner.javaOpts: -server -Xms1g -Xmx1g -XX:MaxDirectMemorySize=1g -Duser.timezone=UTC -Dfile.encoding=UTF-8 -XX:+ExitOnOutOfMemoryError -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.indexer.runner.javaOptsArray: ["-server", "-Xms128m", "-Xmx384m", "-XX:MaxDirectMemorySize=192m", "-Duser.timezone=UTC", "-Dfile.encoding=UTF-8", "-XX:+ExitOnOutOfMemoryError", "-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager"]
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.indexer.task.baseTaskDir: var/druid/task
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.indexer.task.hadoopWorkingPath: var/druid/hadoop-tmp
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.indexing.doubleStorage: double
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.lookup.enableLookupSyncOnStartup: false
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.metadata.storage.connector.connectURI: jdbc:postgresql://druid-int-postgresql:5432/druid
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.metadata.storage.connector.host: localhost
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.metadata.storage.connector.password: <masked>
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.metadata.storage.connector.port: 1527
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.metadata.storage.connector.user: druid
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.metadata.storage.type: postgresql
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.metrics.emitter.dimension.dataSource: DATASOURCE_NAME
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.metrics.emitter.dimension.taskId: index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.metrics.emitter.dimension.taskType: index_kafka
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.monitoring.emissionPeriod: PT2m
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.monitoring.monitors: ["org.apache.druid.java.util.metrics.JvmMonitor"]
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.plaintextPort: 8102
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.processing.buffer.sizeBytes: 2500000
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.processing.numMergeBuffers: 2
2022-01-05T11:50:18,135 INFO [main] org.apache.druid.cli.CliPeon - * druid.processing.numThreads: 1
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.query.groupBy.maxOnDiskStorage: 500000000
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.selectors.coordinator.serviceName: druid/coordinator
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.selectors.indexing.serviceName: druid/overlord
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.server.hiddenProperties: ["druid.s3.accessKey","druid.s3.secretKey","druid.metadata.storage.connector.password"]
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.server.http.defaultQueryTimeout: 3600000
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.server.http.maxIdleTime: PT1h
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.server.http.numThreads: 60
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.service: druid/middleManager
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.sql.enable: true
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.startup.logging.logProperties: true
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.storage.storageDirectory: var/druid/segments
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.storage.type: local
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.task.executor.enablePlaintextPort: true
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.task.executor.enableTlsPort: false
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.task.executor.host: 10.85.6.250
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.task.executor.plaintextPort: 8091
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.task.executor.service: druid/middleManager
2022-01-05T11:50:18,136 INFO [main] org.apache.druid.cli.CliPeon - * druid.task.executor.tlsPort: -1
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * druid.tlsPort: -1
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * druid.worker.capacity: 4
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * druid.zk.paths.base: /druid
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * druid.zk.service.host: zookeeper-int-headless
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * file.encoding: UTF-8
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * file.encoding.pkg: sun.io
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * file.separator: /
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * java.awt.graphicsenv: sun.awt.X11GraphicsEnvironment
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * java.awt.printerjob: sun.print.PSPrinterJob
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * java.class.path: /tmp/conf/druid/cluster/_common:/tmp/conf/druid/cluster/data/middleManager:lib/FastInfoset-1.2.15.jar:lib/RoaringBitmap-0.9.0.jar:lib/accessors-smart-1.2.jar:lib/aether-api-0.9.0.M2.jar:lib/aether-connector-file-0.9.0.M2.jar:lib/aether-connector-okhttp-0.0.9.jar:lib/aether-impl-0.9.0.M2.jar:lib/aether-spi-0.9.0.M2.jar:lib/aether-util-0.9.0.M2.jar:lib/aggdesigner-algorithm-6.0.jar:lib/airline-0.7.jar:lib/antlr4-runtime-4.5.1.jar:lib/aopalliance-1.0.jar:lib/asm-7.1.jar:lib/asm-analysis-7.1.jar:lib/asm-commons-7.1.jar:lib/asm-tree-7.1.jar:lib/async-http-client-2.5.3.jar:lib/async-http-client-netty-utils-2.5.3.jar:lib/audience-annotations-0.5.0.jar:lib/avatica-core-1.17.0.jar:lib/avatica-metrics-1.17.0.jar:lib/avatica-server-1.17.0.jar:lib/aws-java-sdk-core-1.12.37.jar:lib/aws-java-sdk-ec2-1.12.37.jar:lib/aws-java-sdk-kms-1.12.37.jar:lib/aws-java-sdk-s3-1.12.37.jar:lib/caffeine-2.8.0.jar:lib/calcite-
core-1.21.0.jar:lib/calcite-linq4j-1.21.0.jar:lib/checker-qual-2.5.7.jar:lib/classmate-1.1.0.jar:lib/commons-beanutils-1.9.4.jar:lib/commons-codec-1.13.jar:lib/commons-collections-3.2.2.jar:lib/commons-collections4-4.2.jar:lib/commons-compiler-3.0.11.jar:lib/commons-compress-1.21.jar:lib/commons-dbcp2-2.0.1.jar:lib/commons-io-2.11.0.jar:lib/commons-lang-2.6.jar:lib/commons-lang3-3.8.1.jar:lib/commons-logging-1.1.1.jar:lib/commons-math3-3.6.1.jar:lib/commons-net-3.6.jar:lib/commons-pool2-2.2.jar:lib/commons-text-1.3.jar:lib/compress-lzf-1.0.4.jar:lib/config-magic-0.9.jar:lib/cron-scheduler-0.1.jar:lib/curator-client-4.3.0.jar:lib/curator-framework-4.3.0.jar:lib/curator-recipes-4.3.0.jar:lib/curator-x-discovery-4.3.0.jar:lib/datasketches-java-2.0.0.jar:lib/datasketches-memory-1.3.0.jar:lib/derby-10.14.2.0.jar:lib/derbyclient-10.14.2.0.jar:lib/derbynet-10.14.2.0.jar:lib/disruptor-3.3.6.jar:lib/druid-aws-common-0.22.1.jar:lib/druid-console-0.22.1.jar:lib/druid-core-0.22.1.jar:lib/druid-
gcp-common-0.22.1.jar:lib/druid-hll-0.22.1.jar:lib/druid-indexing-hadoop-0.22.1.jar:lib/druid-indexing-service-0.22.1.jar:lib/druid-processing-0.22.1.jar:lib/druid-server-0.22.1.jar:lib/druid-services-0.22.1.jar:lib/druid-sql-0.22.1.jar:lib/error_prone_annotations-2.8.0.jar:lib/esri-geometry-api-2.2.0.jar:lib/extendedset-0.22.1.jar:lib/fastutil-8.5.4.jar:lib/fastutil-core-8.5.4.jar:lib/fastutil-extra-8.5.4.jar:lib/google-api-client-1.26.0.jar:lib/google-http-client-1.26.0.jar:lib/google-http-client-jackson2-1.26.0.jar:lib/google-oauth-client-1.26.0.jar:lib/guava-16.0.1.jar:lib/guice-4.1.0.jar:lib/guice-assistedinject-4.1.0.jar:lib/guice-multibindings-4.1.0.jar:lib/guice-servlet-4.1.0.jar:lib/hibernate-validator-5.2.5.Final.jar:lib/httpclient-4.5.10.jar:lib/httpcore-4.4.11.jar:lib/icu4j-55.1.jar:lib/ion-java-1.0.2.jar:lib/istack-commons-runtime-3.0.7.jar:lib/j2objc-annotations-1.1.jar:lib/jackson-annotations-2.10.5.jar:lib/jackson-core-2.10.5.jar:lib/jackson-databind-2.10.5.1.jar:lib
/jackson-dataformat-cbor-2.10.5.jar:lib/jackson-dataformat-smile-2.10.5.jar:lib/jackson-datatype-guava-2.10.5.jar:lib/jackson-datatype-joda-2.10.5.jar:lib/jackson-jaxrs-base-2.10.5.jar:lib/jackson-jaxrs-json-provider-2.10.5.jar:lib/jackson-jaxrs-smile-provider-2.10.5.jar:lib/jackson-jq-0.0.10.jar:lib/jackson-module-guice-2.10.5.jar:lib/jackson-module-jaxb-annotations-2.10.5.jar:lib/jakarta.activation-api-1.2.1.jar:lib/jakarta.xml.bind-api-2.3.2.jar:lib/janino-3.0.11.jar:lib/javax.activation-1.2.0.jar:lib/javax.activation-api-1.2.0.jar:lib/javax.el-3.0.0.jar:lib/javax.el-api-3.0.0.jar:lib/javax.inject-1.jar:lib/javax.servlet-api-3.1.0.jar:lib/jaxb-api-2.3.1.jar:lib/jaxb-runtime-2.3.1.jar:lib/jboss-logging-3.2.1.Final.jar:lib/jcl-over-slf4j-1.7.12.jar:lib/jcodings-1.0.43.jar:lib/jdbi-2.63.1.jar:lib/jersey-core-1.19.3.jar:lib/jersey-guice-1.19.3.jar:lib/jersey-server-1.19.3.jar:lib/jersey-servlet-1.19.3.jar:lib/jetty-client-9.4.40.v20210413.jar:lib/jetty-continuation-9.4.40.v20210413.j
ar:lib/jetty-http-9.4.40.v20210413.jar:lib/jetty-io-9.4.40.v20210413.jar:lib/jetty-proxy-9.4.40.v20210413.jar:lib/jetty-rewrite-9.4.40.v20210413.jar:lib/jetty-security-9.4.40.v20210413.jar:lib/jetty-server-9.4.40.v20210413.jar:lib/jetty-servlet-9.4.40.v20210413.jar:lib/jetty-servlets-9.4.40.v20210413.jar:lib/jetty-util-9.4.40.v20210413.jar:lib/jetty-util-ajax-9.4.40.v20210413.jar:lib/jmespath-java-1.12.37.jar:lib/jna-4.5.1.jar:lib/joda-time-2.10.5.jar:lib/joni-2.1.27.jar:lib/json-path-2.3.0.jar:lib/json-smart-2.3.jar:lib/jsr305-2.0.1.jar:lib/jsr311-api-1.1.1.jar:lib/jvm-attach-api-1.5.jar:lib/log4j-1.2-api-2.15.0.jar:lib/log4j-api-2.15.0.jar:lib/log4j-core-2.15.0.jar:lib/log4j-jul-2.15.0.jar:lib/log4j-slf4j-impl-2.15.0.jar:lib/lz4-java-1.7.1.jar:lib/maven-aether-provider-3.1.1.jar:lib/maven-artifact-3.6.0.jar:lib/maven-model-3.1.1.jar:lib/maven-model-builder-3.1.1.jar:lib/maven-repository-metadata-3.1.1.jar:lib/maven-settings-3.1.1.jar:lib/maven-settings-builder-3.1.1.jar:lib/metric
s-core-4.0.0.jar:lib/netty-3.10.6.Final.jar:lib/netty-buffer-4.1.68.Final.jar:lib/netty-codec-4.1.68.Final.jar:lib/netty-codec-dns-4.1.68.Final.jar:lib/netty-codec-http-4.1.68.Final.jar:lib/netty-codec-socks-4.1.68.Final.jar:lib/netty-common-4.1.68.Final.jar:lib/netty-handler-4.1.68.Final.jar:lib/netty-handler-proxy-4.1.68.Final.jar:lib/netty-reactive-streams-2.0.0.jar:lib/netty-resolver-4.1.68.Final.jar:lib/netty-resolver-dns-4.1.68.Final.jar:lib/netty-transport-4.1.68.Final.jar:lib/netty-transport-native-epoll-4.1.68.Final-linux-x86_64.jar:lib/netty-transport-native-unix-common-4.1.68.Final.jar:lib/okhttp-1.0.2.jar:lib/opencsv-4.6.jar:lib/plexus-interpolation-1.19.jar:lib/plexus-utils-3.0.24.jar:lib/protobuf-java-3.11.0.jar:lib/reactive-streams-1.0.2.jar:lib/resilience4j-bulkhead-1.3.1.jar:lib/resilience4j-core-1.3.1.jar:lib/rhino-1.7.11.jar:lib/shims-0.9.0.jar:lib/sigar-1.6.5.132.jar:lib/slf4j-api-1.7.25.jar:lib/spymemcached-2.12.3.jar:lib/stax-ex-1.8.jar:lib/tesla-aether-0.0.5.j
ar:lib/txw2-2.3.1.jar:lib/validation-api-1.1.0.Final.jar:lib/vavr-0.10.2.jar:lib/vavr-match-0.10.2.jar:lib/xz-1.8.jar:lib/wagon-provider-api-2.4.jar:lib/zookeeper-3.5.9.jar:lib/zookeeper-jute-3.5.9.jar:lib/zstd-jni-1.3.3-1.jar:
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * java.class.version: 52.0
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * java.endorsed.dirs: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-1.el8_3.x86_64/jre/lib/endorsed
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * java.ext.dirs: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-1.el8_3.x86_64/jre/lib/ext:/usr/java/packages/lib/ext
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-1.el8_3.x86_64/jre
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * java.io.tmpdir: var/tmp
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * java.library.path: /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * java.runtime.name: OpenJDK Runtime Environment
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * java.runtime.version: 1.8.0_275-b01
2022-01-05T11:50:18,137 INFO [main] org.apache.druid.cli.CliPeon - * java.specification.name: Java Platform API Specification
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * java.specification.vendor: Oracle Corporation
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * java.specification.version: 1.8
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * java.util.logging.manager: org.apache.logging.log4j.jul.LogManager
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * java.vendor: Red Hat, Inc.
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * java.vendor.url: https://www.redhat.com/
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * java.vendor.url.bug: https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%208&component=java-1.8.0-openjdk
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * java.version: 1.8.0_275
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * java.vm.info: mixed mode
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * java.vm.name: OpenJDK 64-Bit Server VM
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * java.vm.specification.name: Java Virtual Machine Specification
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * java.vm.specification.vendor: Oracle Corporation
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * java.vm.specification.version: 1.8
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * java.vm.vendor: Red Hat, Inc.
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * java.vm.version: 25.275-b01
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * line.separator:
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * log4j.shutdownCallbackRegistry: org.apache.druid.common.config.Log4jShutdown
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * log4j.shutdownHookEnabled: true
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * log4j2.is.webapp: false
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * os.arch: amd64
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * os.name: Linux
2022-01-05T11:50:18,138 INFO [main] org.apache.druid.cli.CliPeon - * os.version: 4.18.0-305.28.1.el8_4.x86_64
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * path.separator: :
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * sun.arch.data.model: 64
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * sun.boot.class.path: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-1.el8_3.x86_64/jre/lib/resources.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-1.el8_3.x86_64/jre/lib/rt.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-1.el8_3.x86_64/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-1.el8_3.x86_64/jre/lib/jsse.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-1.el8_3.x86_64/jre/lib/jce.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-1.el8_3.x86_64/jre/lib/charsets.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-1.el8_3.x86_64/jre/lib/jfr.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-1.el8_3.x86_64/jre/classes
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * sun.boot.library.path: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-1.el8_3.x86_64/jre/lib/amd64
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * sun.cpu.endian: little
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * sun.cpu.isalist:
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * sun.io.unicode.encoding: UnicodeLittle
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * sun.java.command: org.apache.druid.cli.Main internal peon var/druid/task/index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao/task.json var/druid/task/index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao/d3bb8b64-f303-4e1b-8490-31c18561577a/status.json var/druid/task/index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao/d3bb8b64-f303-4e1b-8490-31c18561577a/report.json --loadBroadcastSegments true
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * sun.java.launcher: SUN_STANDARD
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * sun.jnu.encoding: ANSI_X3.4-1968
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * sun.management.compiler: HotSpot 64-Bit Tiered Compilers
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * sun.os.patch.level: unknown
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * user.country: US
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * user.dir: /opt/druid
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * user.home: ?
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * user.language: en
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * user.name: ?
2022-01-05T11:50:18,139 INFO [main] org.apache.druid.cli.CliPeon - * user.timezone: UTC
2022-01-05T11:50:18,511 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.5.9-83df9301aa5c2a5d284a9940177808c01bc35cef, built on 01/06/2021 20:03 GMT
2022-01-05T11:50:18,511 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:host.name=druid-int-middle-manager-2.druid-int-middle-manager.some_proj_name-global-int.svc.cluster.local
2022-01-05T11:50:18,511 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.8.0_275
2022-01-05T11:50:18,511 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Red Hat, Inc.
2022-01-05T11:50:18,511 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-1.el8_3.x86_64/jre
2022-01-05T11:50:18,511 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=/tmp/conf/druid/cluster/_common:/tmp/conf/druid/cluster/data/middleManager:lib/FastInfoset-1.2.15.jar:lib/RoaringBitmap-0.9.0.jar:lib/accessors-smart-1.2.jar:lib/aether-api-0.9.0.M2.jar:lib/aether-connector-file-0.9.0.M2.jar:lib/aether-connector-okhttp-0.0.9.jar:lib/aether-impl-0.9.0.M2.jar:lib/aether-spi-0.9.0.M2.jar:lib/aether-util-0.9.0.M2.jar:lib/aggdesigner-algorithm-6.0.jar:lib/airline-0.7.jar:lib/antlr4-runtime-4.5.1.jar:lib/aopalliance-1.0.jar:lib/asm-7.1.jar:lib/asm-analysis-7.1.jar:lib/asm-commons-7.1.jar:lib/asm-tree-7.1.jar:lib/async-http-client-2.5.3.jar:lib/async-http-client-netty-utils-2.5.3.jar:lib/audience-annotations-0.5.0.jar:lib/avatica-core-1.17.0.jar:lib/avatica-metrics-1.17.0.jar:lib/avatica-server-1.17.0.jar:lib/aws-java-sdk-core-1.12.37.jar:lib/aws-java-sdk-ec2-1.12.37.jar:lib/aws-java-sdk-kms-1.12.37.jar:lib/aws-java-sdk-s3-1.12.37.jar:lib/caffeine-2.8.
0.jar:lib/calcite-core-1.21.0.jar:lib/calcite-linq4j-1.21.0.jar:lib/checker-qual-2.5.7.jar:lib/classmate-1.1.0.jar:lib/commons-beanutils-1.9.4.jar:lib/commons-codec-1.13.jar:lib/commons-collections-3.2.2.jar:lib/commons-collections4-4.2.jar:lib/commons-compiler-3.0.11.jar:lib/commons-compress-1.21.jar:lib/commons-dbcp2-2.0.1.jar:lib/commons-io-2.11.0.jar:lib/commons-lang-2.6.jar:lib/commons-lang3-3.8.1.jar:lib/commons-logging-1.1.1.jar:lib/commons-math3-3.6.1.jar:lib/commons-net-3.6.jar:lib/commons-pool2-2.2.jar:lib/commons-text-1.3.jar:lib/compress-lzf-1.0.4.jar:lib/config-magic-0.9.jar:lib/cron-scheduler-0.1.jar:lib/curator-client-4.3.0.jar:lib/curator-framework-4.3.0.jar:lib/curator-recipes-4.3.0.jar:lib/curator-x-discovery-4.3.0.jar:lib/datasketches-java-2.0.0.jar:lib/datasketches-memory-1.3.0.jar:lib/derby-10.14.2.0.jar:lib/derbyclient-10.14.2.0.jar:lib/derbynet-10.14.2.0.jar:lib/disruptor-3.3.6.jar:lib/druid-aws-common-0.22.1.jar:lib/druid-console-0.22.1.jar:lib/druid-core-0.2
2.1.jar:lib/druid-gcp-common-0.22.1.jar:lib/druid-hll-0.22.1.jar:lib/druid-indexing-hadoop-0.22.1.jar:lib/druid-indexing-service-0.22.1.jar:lib/druid-processing-0.22.1.jar:lib/druid-server-0.22.1.jar:lib/druid-services-0.22.1.jar:lib/druid-sql-0.22.1.jar:lib/error_prone_annotations-2.8.0.jar:lib/esri-geometry-api-2.2.0.jar:lib/extendedset-0.22.1.jar:lib/fastutil-8.5.4.jar:lib/fastutil-core-8.5.4.jar:lib/fastutil-extra-8.5.4.jar:lib/google-api-client-1.26.0.jar:lib/google-http-client-1.26.0.jar:lib/google-http-client-jackson2-1.26.0.jar:lib/google-oauth-client-1.26.0.jar:lib/guava-16.0.1.jar:lib/guice-4.1.0.jar:lib/guice-assistedinject-4.1.0.jar:lib/guice-multibindings-4.1.0.jar:lib/guice-servlet-4.1.0.jar:lib/hibernate-validator-5.2.5.Final.jar:lib/httpclient-4.5.10.jar:lib/httpcore-4.4.11.jar:lib/icu4j-55.1.jar:lib/ion-java-1.0.2.jar:lib/istack-commons-runtime-3.0.7.jar:lib/j2objc-annotations-1.1.jar:lib/jackson-annotations-2.10.5.jar:lib/jackson-core-2.10.5.jar:lib/jackson-databin
d-2.10.5.1.jar:lib/jackson-dataformat-cbor-2.10.5.jar:lib/jackson-dataformat-smile-2.10.5.jar:lib/jackson-datatype-guava-2.10.5.jar:lib/jackson-datatype-joda-2.10.5.jar:lib/jackson-jaxrs-base-2.10.5.jar:lib/jackson-jaxrs-json-provider-2.10.5.jar:lib/jackson-jaxrs-smile-provider-2.10.5.jar:lib/jackson-jq-0.0.10.jar:lib/jackson-module-guice-2.10.5.jar:lib/jackson-module-jaxb-annotations-2.10.5.jar:lib/jakarta.activation-api-1.2.1.jar:lib/jakarta.xml.bind-api-2.3.2.jar:lib/janino-3.0.11.jar:lib/javax.activation-1.2.0.jar:lib/javax.activation-api-1.2.0.jar:lib/javax.el-3.0.0.jar:lib/javax.el-api-3.0.0.jar:lib/javax.inject-1.jar:lib/javax.servlet-api-3.1.0.jar:lib/jaxb-api-2.3.1.jar:lib/jaxb-runtime-2.3.1.jar:lib/jboss-logging-3.2.1.Final.jar:lib/jcl-over-slf4j-1.7.12.jar:lib/jcodings-1.0.43.jar:lib/jdbi-2.63.1.jar:lib/jersey-core-1.19.3.jar:lib/jersey-guice-1.19.3.jar:lib/jersey-server-1.19.3.jar:lib/jersey-servlet-1.19.3.jar:lib/jetty-client-9.4.40.v20210413.jar:lib/jetty-continuation-
9.4.40.v20210413.jar:lib/jetty-http-9.4.40.v20210413.jar:lib/jetty-io-9.4.40.v20210413.jar:lib/jetty-proxy-9.4.40.v20210413.jar:lib/jetty-rewrite-9.4.40.v20210413.jar:lib/jetty-security-9.4.40.v20210413.jar:lib/jetty-server-9.4.40.v20210413.jar:lib/jetty-servlet-9.4.40.v20210413.jar:lib/jetty-servlets-9.4.40.v20210413.jar:lib/jetty-util-9.4.40.v20210413.jar:lib/jetty-util-ajax-9.4.40.v20210413.jar:lib/jmespath-java-1.12.37.jar:lib/jna-4.5.1.jar:lib/joda-time-2.10.5.jar:lib/joni-2.1.27.jar:lib/json-path-2.3.0.jar:lib/json-smart-2.3.jar:lib/jsr305-2.0.1.jar:lib/jsr311-api-1.1.1.jar:lib/jvm-attach-api-1.5.jar:lib/log4j-1.2-api-2.15.0.jar:lib/log4j-api-2.15.0.jar:lib/log4j-core-2.15.0.jar:lib/log4j-jul-2.15.0.jar:lib/log4j-slf4j-impl-2.15.0.jar:lib/lz4-java-1.7.1.jar:lib/maven-aether-provider-3.1.1.jar:lib/maven-artifact-3.6.0.jar:lib/maven-model-3.1.1.jar:lib/maven-model-builder-3.1.1.jar:lib/maven-repository-metadata-3.1.1.jar:lib/maven-settings-3.1.1.jar:lib/maven-settings-builder-3.
1.1.jar:lib/metrics-core-4.0.0.jar:lib/netty-3.10.6.Final.jar:lib/netty-buffer-4.1.68.Final.jar:lib/netty-codec-4.1.68.Final.jar:lib/netty-codec-dns-4.1.68.Final.jar:lib/netty-codec-http-4.1.68.Final.jar:lib/netty-codec-socks-4.1.68.Final.jar:lib/netty-common-4.1.68.Final.jar:lib/netty-handler-4.1.68.Final.jar:lib/netty-handler-proxy-4.1.68.Final.jar:lib/netty-reactive-streams-2.0.0.jar:lib/netty-resolver-4.1.68.Final.jar:lib/netty-resolver-dns-4.1.68.Final.jar:lib/netty-transport-4.1.68.Final.jar:lib/netty-transport-native-epoll-4.1.68.Final-linux-x86_64.jar:lib/netty-transport-native-unix-common-4.1.68.Final.jar:lib/okhttp-1.0.2.jar:lib/opencsv-4.6.jar:lib/plexus-interpolation-1.19.jar:lib/plexus-utils-3.0.24.jar:lib/protobuf-java-3.11.0.jar:lib/reactive-streams-1.0.2.jar:lib/resilience4j-bulkhead-1.3.1.jar:lib/resilience4j-core-1.3.1.jar:lib/rhino-1.7.11.jar:lib/shims-0.9.0.jar:lib/sigar-1.6.5.132.jar:lib/slf4j-api-1.7.25.jar:lib/spymemcached-2.12.3.jar:lib/stax-ex-1.8.jar:lib/te
sla-aether-0.0.5.jar:lib/txw2-2.3.1.jar:lib/validation-api-1.1.0.Final.jar:lib/vavr-0.10.2.jar:lib/vavr-match-0.10.2.jar:lib/xz-1.8.jar:lib/wagon-provider-api-2.4.jar:lib/zookeeper-3.5.9.jar:lib/zookeeper-jute-3.5.9.jar:lib/zstd-jni-1.3.3-1.jar:
2022-01-05T11:50:18,511 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2022-01-05T11:50:18,511 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=var/tmp
2022-01-05T11:50:18,511 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
2022-01-05T11:50:18,511 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux
2022-01-05T11:50:18,511 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
2022-01-05T11:50:18,511 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.version=4.18.0-305.28.1.el8_4.x86_64
2022-01-05T11:50:18,512 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:user.name=?
2022-01-05T11:50:18,512 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:user.home=?
2022-01-05T11:50:18,512 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/opt/druid
2022-01-05T11:50:18,512 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.memory.free=127MB
2022-01-05T11:50:18,512 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.memory.max=341MB
2022-01-05T11:50:18,512 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.memory.total=217MB
2022-01-05T11:50:18,512 INFO [main] org.apache.curator.utils.Compatibility - Using emulated InjectSessionExpiration
2022-01-05T11:50:18,552 WARN [main] org.apache.druid.indexing.common.config.TaskConfig - Batch processing mode argument value is null or not valid:[null], defaulting to[CLOSED_SEGMENTS]
2022-01-05T11:50:18,552 INFO [main] org.apache.druid.indexing.common.config.TaskConfig - Batch processing mode:[CLOSED_SEGMENTS]
2022-01-05T11:50:18,653 INFO [main] org.apache.druid.segment.loading.SegmentLocalCacheManager - Using storage location strategy: [LeastBytesUsedStorageLocationSelectorStrategy]
2022-01-05T11:50:18,708 INFO [main] org.apache.druid.guice.StorageNodeModule - Segment cache not configured on ServerType [indexer-executor]. It will not be assignable for segment placement
2022-01-05T11:50:18,768 INFO [main] org.eclipse.jetty.util.log - Logging initialized @3742ms to org.eclipse.jetty.util.log.Slf4jLog
2022-01-05T11:50:18,781 INFO [main] org.apache.druid.server.initialization.jetty.JettyServerModule - Creating http connector with port [8102]
2022-01-05T11:50:18,932 WARN [main] org.eclipse.jetty.server.handler.gzip.GzipHandler - minGzipSize of 0 is inefficient for short content, break even is size 23
2022-01-05T11:50:18,961 INFO [main] org.apache.druid.offheap.OffheapBufferGenerator - Allocating new intermediate processing buffer[0] of size[2,500,000]
2022-01-05T11:50:19,036 INFO [main] org.apache.druid.offheap.OffheapBufferGenerator - Allocating new result merging buffer[0] of size[2,500,000]
2022-01-05T11:50:19,037 INFO [main] org.apache.druid.offheap.OffheapBufferGenerator - Allocating new result merging buffer[1] of size[2,500,000]
2022-01-05T11:50:19,047 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Starting lifecycle [module] stage [INIT]
2022-01-05T11:50:19,047 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Starting lifecycle [module] stage [NORMAL]
2022-01-05T11:50:19,051 INFO [main] org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
2022-01-05T11:50:19,056 INFO [main] org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=zookeeper-int-headless sessionTimeout=30000 watcher=org.apache.curator.ConnectionState@7185e611
2022-01-05T11:50:19,108 INFO [main] org.apache.zookeeper.common.X509Util - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
2022-01-05T11:50:19,112 INFO [main] org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 4194304 Bytes
2022-01-05T11:50:19,120 INFO [main] org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=
2022-01-05T11:50:19,127 INFO [main] org.apache.curator.framework.imps.CuratorFrameworkImpl - Default schema
2022-01-05T11:50:19,129 INFO [main-SendThread(zookeeper-int-headless:2181)] org.apache.zookeeper.ClientCnxn - Opening socket connection to server zookeeper-int-headless/10.84.7.11:2181. Will not attempt to authenticate using SASL (unknown error)
2022-01-05T11:50:19,135 INFO [main] org.apache.druid.security.basic.authentication.db.cache.CoordinatorPollingBasicAuthenticatorCacheManager - Starting CoordinatorPollingBasicAuthenticatorCacheManager.
2022-01-05T11:50:19,136 INFO [main-SendThread(zookeeper-int-headless:2181)] org.apache.zookeeper.ClientCnxn - Socket connection established, initiating session, client: /10.85.6.250:46610, server: zookeeper-int-headless/10.84.7.11:2181
2022-01-05T11:50:19,145 INFO [main-SendThread(zookeeper-int-headless:2181)] org.apache.zookeeper.ClientCnxn - Session establishment complete on server zookeeper-int-headless/10.84.7.11:2181, sessionid = 0x20006c33aae8561, negotiated timeout = 30000
2022-01-05T11:50:19,148 INFO [main-EventThread] org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED
2022-01-05T11:50:19,208 INFO [main-EventThread] org.apache.curator.framework.imps.EnsembleTracker - New config event received: {server.1=zookeeper-int-0.zookeeper-int-headless.some_proj_name-global-int.svc.cluster.local:2888:3888:participant;0.0.0.0:2181, version=0, server.3=zookeeper-int-2.zookeeper-int-headless.some_proj_name-global-int.svc.cluster.local:2888:3888:participant;0.0.0.0:2181, server.2=zookeeper-int-1.zookeeper-int-headless.some_proj_name-global-int.svc.cluster.local:2888:3888:participant;0.0.0.0:2181}
2022-01-05T11:50:19,216 WARN [NodeRoleWatcher[COORDINATOR]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeRoleWatcher - Ignored event type[CONNECTION_RECONNECTED] for node watcher of role[coordinator].
2022-01-05T11:50:19,221 INFO [main-EventThread] org.apache.curator.framework.imps.EnsembleTracker - New config event received: {server.1=zookeeper-int-0.zookeeper-int-headless.some_proj_name-global-int.svc.cluster.local:2888:3888:participant;0.0.0.0:2181, version=0, server.3=zookeeper-int-2.zookeeper-int-headless.some_proj_name-global-int.svc.cluster.local:2888:3888:participant;0.0.0.0:2181, server.2=zookeeper-int-1.zookeeper-int-headless.some_proj_name-global-int.svc.cluster.local:2888:3888:participant;0.0.0.0:2181}
2022-01-05T11:50:19,238 INFO [NodeRoleWatcher[COORDINATOR]] org.apache.druid.discovery.BaseNodeRoleWatcher - Node[http://10.85.5.178:8081] of role[coordinator] detected.
2022-01-05T11:50:19,238 INFO [NodeRoleWatcher[COORDINATOR]] org.apache.druid.discovery.BaseNodeRoleWatcher - Node watcher of role[coordinator] is now initialized.
2022-01-05T11:50:19,545 INFO [main] org.apache.druid.security.basic.authentication.db.cache.CoordinatorPollingBasicAuthenticatorCacheManager - Started CoordinatorPollingBasicAuthenticatorCacheManager.
2022-01-05T11:50:19,545 INFO [main] org.apache.druid.security.basic.authorization.db.cache.CoordinatorPollingBasicAuthorizerCacheManager - Starting CoordinatorPollingBasicAuthorizerCacheManager.
2022-01-05T11:50:19,663 INFO [main] org.apache.druid.security.basic.authorization.db.cache.CoordinatorPollingBasicAuthorizerCacheManager - Started CoordinatorPollingBasicAuthorizerCacheManager.
2022-01-05T11:50:19,675 INFO [NodeRoleWatcher[OVERLORD]] org.apache.druid.discovery.BaseNodeRoleWatcher - Node[http://10.85.5.178:8081] of role[overlord] detected.
2022-01-05T11:50:19,710 INFO [NodeRoleWatcher[OVERLORD]] org.apache.druid.discovery.BaseNodeRoleWatcher - Node watcher of role[overlord] is now initialized.
2022-01-05T11:50:19,814 INFO [main] org.apache.druid.indexing.worker.executor.ExecutorLifecycle - Running with task: {
"type" : "index_kafka",
"id" : "index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao",
"resource" : {
"availabilityGroup" : "index_kafka_DATASOURCE_NAME_11fc40b96a318a6",
"requiredCapacity" : 1
},
"dataSchema" : {
"dataSource" : "DATASOURCE_NAME",
"timestampSpec" : {
"column" : "CREATEDAT",
"format" : "auto",
"missingValue" : null
},
"dimensionsSpec" : {
"dimensions" : [ **ommited** ],
"dimensionExclusions" : [ "__time" ]
},
"metricsSpec" : [ ],
"granularitySpec" : {
"type" : "uniform",
"segmentGranularity" : "MONTH",
"queryGranularity" : {
"type" : "none"
},
"rollup" : false,
"intervals" : [ ]
},
"transformSpec" : {
"filter" : null,
"transforms" : [ ]
}
},
"tuningConfig" : {
"type" : "KafkaTuningConfig",
"appendableIndexSpec" : {
"type" : "onheap"
},
"maxRowsInMemory" : 1000000,
"maxBytesInMemory" : 0,
"skipBytesInMemoryOverheadCheck" : false,
"maxRowsPerSegment" : 5000000,
"maxTotalRows" : null,
"intermediatePersistPeriod" : "PT60S",
"basePersistDirectory" : "/opt/druid/var/tmp/druid-realtime-persist1014422271737391165",
"maxPendingPersists" : 0,
"indexSpec" : {
"bitmap" : {
"type" : "roaring",
"compressRunOnSerialization" : true
},
"dimensionCompression" : "lz4",
"metricCompression" : "lz4",
"longEncoding" : "longs",
"segmentLoader" : null
},
"indexSpecForIntermediatePersists" : {
"bitmap" : {
"type" : "roaring",
"compressRunOnSerialization" : true
},
"dimensionCompression" : "lz4",
"metricCompression" : "lz4",
"longEncoding" : "longs",
"segmentLoader" : null
},
"reportParseExceptions" : false,
"handoffConditionTimeout" : 0,
"resetOffsetAutomatically" : false,
"segmentWriteOutMediumFactory" : null,
"intermediateHandoffPeriod" : "P2147483647D",
"logParseExceptions" : true,
"maxParseExceptions" : 2147483647,
"maxSavedParseExceptions" : 100,
"skipSequenceNumberAvailabilityCheck" : false
},
"ioConfig" : {
"type" : "kafka",
"taskGroupId" : 0,
"baseSequenceName" : "index_kafka_DATASOURCE_NAME_11fc40b96a318a6",
"startPartitions" : {
"type" : "end",
"stream" : "DATASOURCE_NAME",
"topic" : "DATASOURCE_NAME",
"partitionSequenceNumberMap" : {
"0" : 39476
},
"partitionOffsetMap" : {
"0" : 39476
}
},
"endPartitions" : {
"type" : "end",
"stream" : "DATASOURCE_NAME",
"topic" : "DATASOURCE_NAME",
"partitionSequenceNumberMap" : {
"0" : 9223372036854775807
},
"partitionOffsetMap" : {
"0" : 9223372036854775807
}
},
"startSequenceNumbers" : {
"type" : "start",
"stream" : "DATASOURCE_NAME",
"topic" : "DATASOURCE_NAME",
"partitionSequenceNumberMap" : {
"0" : 39476
},
"partitionOffsetMap" : {
"0" : 39476
},
"exclusivePartitions" : [ ]
},
"endSequenceNumbers" : {
"type" : "end",
"stream" : "DATASOURCE_NAME",
"topic" : "DATASOURCE_NAME",
"partitionSequenceNumberMap" : {
"0" : 9223372036854775807
},
"partitionOffsetMap" : {
"0" : 9223372036854775807
}
},
"consumerProperties" : {
"bootstrap.servers" : "kafka-int-kafka-bootstrap.some_proj_name-global-int.svc.cluster.local:9093",
"security.protocol" : "SSL",
"ssl.truststore.location" : "/pki/ca/ca.p12",
"ssl.truststore.password" : {
"type" : "environment",
"variable" : "KAFKA_CLUSTER_CA_PASSWORD"
},
"ssl.keystore.location" : "/pki/user/user.p12",
"ssl.keystore.password" : {
"type" : "environment",
"variable" : "default_user"
},
"ssl.key.password" : {
"type" : "environment",
"variable" : "default_user"
}
},
"pollTimeout" : 100,
"useTransaction" : true,
"minimumMessageTime" : null,
"maximumMessageTime" : null,
"inputFormat" : {
"type" : "json",
"flattenSpec" : {
"useFieldDiscovery" : true,
"fields" : [ ]
},
"featureSpec" : { }
}
},
"context" : {
"checkpoints" : "{\"0\":{\"0\":39476}}",
"useLineageBasedSegmentAllocation" : true,
"IS_INCREMENTAL_HANDOFF_SUPPORTED" : true,
"forceTimeChunkLock" : true
},
"groupId" : "index_kafka_DATASOURCE_NAME",
"dataSource" : "DATASOURCE_NAME"
}
2022-01-05T11:50:19,815 INFO [main] org.apache.druid.indexing.worker.executor.ExecutorLifecycle - Attempting to lock file[var/druid/task/index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao/lock].
2022-01-05T11:50:19,817 INFO [main] org.apache.druid.indexing.worker.executor.ExecutorLifecycle - Acquired lock file[var/druid/task/index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao/lock] in 1ms.
2022-01-05T11:50:19,824 INFO [main] org.apache.druid.segment.loading.SegmentLocalCacheManager - Using storage location strategy: [LeastBytesUsedStorageLocationSelectorStrategy]
2022-01-05T11:50:19,830 INFO [task-runner-0-priority-0] org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner - Running task: index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao
2022-01-05T11:50:19,831 INFO [main] org.apache.druid.server.coordination.SegmentLoadDropHandler - Starting...
2022-01-05T11:50:19,831 INFO [main] org.apache.druid.server.coordination.CuratorDataSegmentServerAnnouncer - Announcing self[DruidServerMetadata{name='10.85.6.250:8102', hostAndPort='10.85.6.250:8102', hostAndTlsPort='null', maxSize=0, tier='_default_tier', type=indexer-executor, priority=0}] at [/druid/announcements/10.85.6.250:8102]
2022-01-05T11:50:19,834 INFO [main] org.apache.druid.server.coordination.SegmentLoadDropHandler - Started.
2022-01-05T11:50:19,834 INFO [main] org.apache.druid.server.coordination.ZkCoordinator - Starting zkCoordinator for server[10.85.6.250:8102]
2022-01-05T11:50:19,902 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Starting lifecycle [module] stage [SERVER]
2022-01-05T11:50:19,906 INFO [main] org.eclipse.jetty.server.Server - jetty-9.4.40.v20210413; built: 2021-04-13T20:42:42.668Z; git: b881a572662e1943a14ae12e7e1207989f218b74; jvm 1.8.0_275-b01
2022-01-05T11:50:19,917 INFO [task-runner-0-priority-0] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Starting with sequences: [SequenceMetadata{sequenceId=0, sequenceName='index_kafka_DATASOURCE_NAME_11fc40b96a318a6_0', assignments=[0], startOffsets={0=39476}, exclusiveStartPartitions=[], endOffsets={0=9223372036854775807}, sentinel=false, checkpointed=false}]
2022-01-05T11:50:20,005 INFO [main] org.eclipse.jetty.server.session - DefaultSessionIdManager workerName=node0
2022-01-05T11:50:20,005 INFO [main] org.eclipse.jetty.server.session - No SessionScavenger set, using defaults
2022-01-05T11:50:20,006 INFO [main] org.eclipse.jetty.server.session - node0 Scavenging every 600000ms
2022-01-05T11:50:20,028 INFO [task-runner-0-priority-0] org.apache.kafka.clients.consumer.ConsumerConfig - ConsumerConfig values:
allow.auto.create.topics = true
auto.commit.interval.ms = 5000
auto.offset.reset = none
bootstrap.servers = [kafka-int-kafka-bootstrap.some_proj_name-global-int.svc.cluster.local:9093]
check.crcs = true
client.dns.lookup = use_all_dns_ips
client.id = consumer-kafka-supervisor-knonnadg-1
client.rack =
connections.max.idle.ms = 540000
default.api.timeout.ms = 60000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id = kafka-supervisor-knonnadg
group.instance.id = null
heartbeat.interval.ms = 3000
interceptor.classes = []
internal.leave.group.on.close = true
internal.throw.on.fetch.stable.offset.unsupported = false
isolation.level = read_committed
key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 300000
max.poll.records = 500
metadata.max.age.ms = 10000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = SSL
security.providers = null
send.buffer.bytes = 131072
session.timeout.ms = 10000
socket.connection.setup.timeout.max.ms = 30000
socket.connection.setup.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2]
ssl.endpoint.identification.algorithm = https
ssl.engine.factory.class = null
ssl.key.password = [hidden]
ssl.keymanager.algorithm = SunX509
ssl.keystore.certificate.chain = null
ssl.keystore.key = null
ssl.keystore.location = /pki/user/user.p12
ssl.keystore.password = [hidden]
ssl.keystore.type = JKS
ssl.protocol = TLSv1.2
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.certificates = null
ssl.truststore.location = /pki/ca/ca.p12
ssl.truststore.password = [hidden]
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
2022-01-05T11:50:20,128 INFO [main] com.sun.jersey.server.impl.application.WebApplicationImpl - Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:43 PM'
2022-01-05T11:50:20,820 INFO [task-runner-0-priority-0] org.apache.kafka.common.utils.AppInfoParser - Kafka version: 2.8.0
2022-01-05T11:50:20,820 INFO [task-runner-0-priority-0] org.apache.kafka.common.utils.AppInfoParser - Kafka commitId: ebb1d6e21cc92130
2022-01-05T11:50:20,820 INFO [task-runner-0-priority-0] org.apache.kafka.common.utils.AppInfoParser - Kafka startTimeMs: 1641383420818
2022-01-05T11:50:20,830 INFO [task-runner-0-priority-0] org.apache.druid.curator.discovery.CuratorDruidNodeAnnouncer - Announced self [{"druidNode":{"service":"druid/middleManager","host":"10.85.6.250","bindOnHost":false,"plaintextPort":8102,"port":-1,"tlsPort":-1,"enablePlaintextPort":true,"enableTlsPort":false},"nodeType":"peon","services":{"dataNodeService":{"type":"dataNodeService","tier":"_default_tier","maxSize":0,"type":"indexer-executor","priority":0},"lookupNodeService":{"type":"lookupNodeService","lookupTier":"__default"}}}].
2022-01-05T11:50:20,930 INFO [task-runner-0-priority-0] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Initialized sequences: SequenceMetadata{sequenceId=0, sequenceName='index_kafka_DATASOURCE_NAME_11fc40b96a318a6_0', assignments=[0], startOffsets={0=39476}, exclusiveStartPartitions=[], endOffsets={0=9223372036854775807}, sentinel=false, checkpointed=false}
2022-01-05T11:50:20,931 INFO [task-runner-0-priority-0] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Adding partition[0], start[39476] -> end[9223372036854775807] to assignment.
2022-01-05T11:50:20,933 INFO [task-runner-0-priority-0] org.apache.kafka.clients.consumer.KafkaConsumer - [Consumer clientId=consumer-kafka-supervisor-knonnadg-1, groupId=kafka-supervisor-knonnadg] Subscribed to partition(s): DATASOURCE_NAME-0
2022-01-05T11:50:20,937 INFO [task-runner-0-priority-0] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Seeking partition[0] to[39476].
2022-01-05T11:50:20,937 INFO [task-runner-0-priority-0] org.apache.kafka.clients.consumer.KafkaConsumer - [Consumer clientId=consumer-kafka-supervisor-knonnadg-1, groupId=kafka-supervisor-knonnadg] Seeking to offset 39476 for partition DATASOURCE_NAME-0
2022-01-05T11:50:21,411 INFO [main] org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@38826984{/,null,AVAILABLE}
2022-01-05T11:50:21,421 INFO [main] org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@3f1d6a13{HTTP/1.1, (http/1.1)}{0.0.0.0:8102}
2022-01-05T11:50:21,421 INFO [main] org.eclipse.jetty.server.Server - Started @6397ms
2022-01-05T11:50:21,422 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Starting lifecycle [module] stage [ANNOUNCEMENTS]
2022-01-05T11:50:21,445 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Successfully started lifecycle [module]
2022-01-05T11:50:22,245 INFO [task-runner-0-priority-0] org.apache.kafka.clients.Metadata - [Consumer clientId=consumer-kafka-supervisor-knonnadg-1, groupId=kafka-supervisor-knonnadg] Cluster ID: LCOjcmu5S4Cfusr5J4ji5A
2022-01-05T11:50:22,403 INFO [task-runner-0-priority-0] org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-kafka-supervisor-knonnadg-1, groupId=kafka-supervisor-knonnadg] Fetch position FetchPosition{offset=39476, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[kafka-int-kafka-2.kafka-int-kafka-brokers.some_proj_name-global-int.svc:9093 (id: 2 rack: null)], epoch=8}} is out of range for partition DATASOURCE_NAME-0, raising error to the application since no reset policy is configured
2022-01-05T11:50:22,404 WARN [task-runner-0-priority-0] org.apache.druid.indexing.kafka.IncrementalPublishingKafkaIndexTaskRunner - OffsetOutOfRangeException with message [Fetch position FetchPosition{offset=39476, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[kafka-int-kafka-2.kafka-int-kafka-brokers.some_proj_name-global-int.svc:9093 (id: 2 rack: null)], epoch=8}} is out of range for partition DATASOURCE_NAME-0]
2022-01-05T11:50:22,404 WARN [task-runner-0-priority-0] org.apache.druid.indexing.kafka.IncrementalPublishingKafkaIndexTaskRunner - Retrying in 30000ms
2022-01-05T11:50:52,407 INFO [task-runner-0-priority-0] org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-kafka-supervisor-knonnadg-1, groupId=kafka-supervisor-knonnadg] Fetch position FetchPosition{offset=39476, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[kafka-int-kafka-2.kafka-int-kafka-brokers.some_proj_name-global-int.svc:9093 (id: 2 rack: null)], epoch=8}} is out of range for partition DATASOURCE_NAME-0, raising error to the application since no reset policy is configured
2022-01-05T11:50:52,407 WARN [task-runner-0-priority-0] org.apache.druid.indexing.kafka.IncrementalPublishingKafkaIndexTaskRunner - OffsetOutOfRangeException with message [Fetch position FetchPosition{offset=39476, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[kafka-int-kafka-2.kafka-int-kafka-brokers.some_proj_name-global-int.svc:9093 (id: 2 rack: null)], epoch=8}} is out of range for partition DATASOURCE_NAME-0]
2022-01-05T11:50:52,408 WARN [task-runner-0-priority-0] org.apache.druid.indexing.kafka.IncrementalPublishingKafkaIndexTaskRunner - Retrying in 30000ms
2022-01-05T11:51:22,411 INFO [task-runner-0-priority-0] org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-kafka-supervisor-knonnadg-1, groupId=kafka-supervisor-knonnadg] Fetch position FetchPosition{offset=39476, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[kafka-int-kafka-2.kafka-int-kafka-brokers.some_proj_name-global-int.svc:9093 (id: 2 rack: null)], epoch=8}} is out of range for partition DATASOURCE_NAME-0, raising error to the application since no reset policy is configured
2022-01-05T11:51:22,411 WARN [task-runner-0-priority-0] org.apache.druid.indexing.kafka.IncrementalPublishingKafkaIndexTaskRunner - OffsetOutOfRangeException with message [Fetch position FetchPosition{offset=39476, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[kafka-int-kafka-2.kafka-int-kafka-brokers.some_proj_name-global-int.svc:9093 (id: 2 rack: null)], epoch=8}} is out of range for partition DATASOURCE_NAME-0]
2022-01-05T11:51:22,411 WARN [task-runner-0-priority-0] org.apache.druid.indexing.kafka.IncrementalPublishingKafkaIndexTaskRunner - Retrying in 30000ms
2022-01-05T11:51:52,414 INFO [task-runner-0-priority-0] org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-kafka-supervisor-knonnadg-1, groupId=kafka-supervisor-knonnadg] Fetch position FetchPosition{offset=39476, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[kafka-int-kafka-2.kafka-int-kafka-brokers.some_proj_name-global-int.svc:9093 (id: 2 rack: null)], epoch=8}} is out of range for partition DATASOURCE_NAME-0, raising error to the application since no reset policy is configured
2022-01-05T11:51:52,414 WARN [task-runner-0-priority-0] org.apache.druid.indexing.kafka.IncrementalPublishingKafkaIndexTaskRunner - OffsetOutOfRangeException with message [Fetch position FetchPosition{offset=39476, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[kafka-int-kafka-2.kafka-int-kafka-brokers.some_proj_name-global-int.svc:9093 (id: 2 rack: null)], epoch=8}} is out of range for partition DATASOURCE_NAME-0]
2022-01-05T11:51:52,414 WARN [task-runner-0-priority-0] org.apache.druid.indexing.kafka.IncrementalPublishingKafkaIndexTaskRunner - Retrying in 30000ms
2022-01-05T11:52:22,418 INFO [task-runner-0-priority-0] org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-kafka-supervisor-knonnadg-1, groupId=kafka-supervisor-knonnadg] Fetch position FetchPosition{offset=39476, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[kafka-int-kafka-2.kafka-int-kafka-brokers.some_proj_name-global-int.svc:9093 (id: 2 rack: null)], epoch=8}} is out of range for partition DATASOURCE_NAME-0, raising error to the application since no reset policy is configured
2022-01-05T11:52:22,418 WARN [task-runner-0-priority-0] org.apache.druid.indexing.kafka.IncrementalPublishingKafkaIndexTaskRunner - OffsetOutOfRangeException with message [Fetch position FetchPosition{offset=39476, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[kafka-int-kafka-2.kafka-int-kafka-brokers.some_proj_name-global-int.svc:9093 (id: 2 rack: null)], epoch=8}} is out of range for partition DATASOURCE_NAME-0]
2022-01-05T11:52:22,418 WARN [task-runner-0-priority-0] org.apache.druid.indexing.kafka.IncrementalPublishingKafkaIndexTaskRunner - Retrying in 30000ms
2022-01-05T11:52:52,421 INFO [task-runner-0-priority-0] org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-kafka-supervisor-knonnadg-1, groupId=kafka-supervisor-knonnadg] Fetch position FetchPosition{offset=39476, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[kafka-int-kafka-2.kafka-int-kafka-brokers.some_proj_name-global-int.svc:9093 (id: 2 rack: null)], epoch=8}} is out of range for partition DATASOURCE_NAME-0, raising error to the application since no reset policy is configured
2022-01-05T11:52:52,421 WARN [task-runner-0-priority-0] org.apache.druid.indexing.kafka.IncrementalPublishingKafkaIndexTaskRunner - OffsetOutOfRangeException with message [Fetch position FetchPosition{offset=39476, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[kafka-int-kafka-2.kafka-int-kafka-brokers.some_proj_name-global-int.svc:9093 (id: 2 rack: null)], epoch=8}} is out of range for partition DATASOURCE_NAME-0]
2022-01-05T11:52:52,421 WARN [task-runner-0-priority-0] org.apache.druid.indexing.kafka.IncrementalPublishingKafkaIndexTaskRunner - Retrying in 30000ms
2022-01-05T11:53:20,937 INFO [qtp1010040056-126] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Sequence[index_kafka_DATASOURCE_NAME_11fc40b96a318a6_0] end offsets updated from [{0=9223372036854775807}] to [{0=39476}].
2022-01-05T11:53:20,944 INFO [qtp1010040056-126] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Saved sequence metadata to disk: [SequenceMetadata{sequenceId=0, sequenceName='index_kafka_DATASOURCE_NAME_11fc40b96a318a6_0', assignments=[0], startOffsets={0=39476}, exclusiveStartPartitions=[], endOffsets={0=39476}, sentinel=false, checkpointed=true}]
2022-01-05T11:53:20,944 INFO [task-runner-0-priority-0] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Finished reading partition[0], up to[39476].
2022-01-05T11:53:20,944 INFO [task-runner-0-priority-0] org.apache.kafka.clients.consumer.KafkaConsumer - [Consumer clientId=consumer-kafka-supervisor-knonnadg-1, groupId=kafka-supervisor-knonnadg] Unsubscribed all topics or patterns and assigned partitions
2022-01-05T11:53:20,949 INFO [task-runner-0-priority-0] org.apache.druid.segment.realtime.appenderator.StreamAppenderator - Persisted rows[0] and (estimated) bytes[0]
2022-01-05T11:53:20,955 INFO [[index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.StreamAppenderator - Flushed in-memory data with commit metadata [AppenderatorDriverMetadata{segments={}, lastSegmentIds={}, callerMetadata={nextPartitions=SeekableStreamEndSequenceNumbers{stream='DATASOURCE_NAME', partitionSequenceNumberMap={0=39476}}}}] for segments:
2022-01-05T11:53:20,955 INFO [[index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.StreamAppenderator - Persisted stats: processed rows: [0], persisted rows[0], sinks: [0], total fireHydrants (across sinks): [0], persisted fireHydrants (across sinks): [0]
2022-01-05T11:53:20,958 INFO [task-runner-0-priority-0] org.apache.druid.segment.realtime.appenderator.BaseAppenderatorDriver - Pushing [0] segments in background
2022-01-05T11:53:20,960 INFO [task-runner-0-priority-0] org.apache.druid.segment.realtime.appenderator.StreamAppenderator - Persisted rows[0] and (estimated) bytes[0]
2022-01-05T11:53:20,967 INFO [[index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.StreamAppenderator - Flushed in-memory data with commit metadata [AppenderatorDriverMetadata{segments={}, lastSegmentIds={}, callerMetadata={nextPartitions=SeekableStreamStartSequenceNumbers{stream='DATASOURCE_NAME', partitionSequenceNumberMap={0=39476}, exclusivePartitions=[]}, publishPartitions=SeekableStreamEndSequenceNumbers{stream='DATASOURCE_NAME', partitionSequenceNumberMap={0=39476}}}}] for segments:
2022-01-05T11:53:20,968 INFO [[index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.StreamAppenderator - Persisted stats: processed rows: [0], persisted rows[0], sinks: [0], total fireHydrants (across sinks): [0], persisted fireHydrants (across sinks): [0]
2022-01-05T11:53:20,968 INFO [[index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao]-appenderator-merge] org.apache.druid.segment.realtime.appenderator.StreamAppenderator - Preparing to push (stats): processed rows: [0], sinks: [0], fireHydrants (across sinks): [0]
2022-01-05T11:53:20,969 INFO [[index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao]-appenderator-merge] org.apache.druid.segment.realtime.appenderator.StreamAppenderator - Push complete...
2022-01-05T11:53:20,975 INFO [[index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao]-publish] org.apache.druid.indexing.seekablestream.SequenceMetadata - With empty segment set, start offsets [SeekableStreamStartSequenceNumbers{stream='DATASOURCE_NAME', partitionSequenceNumberMap={0=39476}, exclusivePartitions=[]}] and end offsets [SeekableStreamEndSequenceNumbers{stream='DATASOURCE_NAME', partitionSequenceNumberMap={0=39476}}] are the same, skipping metadata commit.
2022-01-05T11:53:20,976 INFO [[index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao]-publish] org.apache.druid.segment.realtime.appenderator.BaseAppenderatorDriver - Published [0] segments with commit metadata [{nextPartitions=SeekableStreamStartSequenceNumbers{stream='DATASOURCE_NAME', partitionSequenceNumberMap={0=39476}, exclusivePartitions=[]}, publishPartitions=SeekableStreamEndSequenceNumbers{stream='DATASOURCE_NAME', partitionSequenceNumberMap={0=39476}}}]
2022-01-05T11:53:20,977 INFO [[index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao]-publish] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Published 0 segments for sequence [index_kafka_DATASOURCE_NAME_11fc40b96a318a6_0] with metadata [AppenderatorDriverMetadata{segments={}, lastSegmentIds={}, callerMetadata={nextPartitions=SeekableStreamStartSequenceNumbers{stream='DATASOURCE_NAME', partitionSequenceNumberMap={0=39476}, exclusivePartitions=[]}, publishPartitions=SeekableStreamEndSequenceNumbers{stream='DATASOURCE_NAME', partitionSequenceNumberMap={0=39476}}}}].
2022-01-05T11:53:20,980 INFO [[index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao]-publish] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Saved sequence metadata to disk: []
2022-01-05T11:53:20,982 INFO [task-runner-0-priority-0] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Handoff complete for segments:
2022-01-05T11:53:20,984 INFO [task-runner-0-priority-0] org.apache.kafka.common.metrics.Metrics - Metrics scheduler closed
2022-01-05T11:53:20,984 INFO [task-runner-0-priority-0] org.apache.kafka.common.metrics.Metrics - Closing reporter org.apache.kafka.common.metrics.JmxReporter
2022-01-05T11:53:20,984 INFO [task-runner-0-priority-0] org.apache.kafka.common.metrics.Metrics - Metrics reporters closed
2022-01-05T11:53:20,988 INFO [task-runner-0-priority-0] org.apache.kafka.common.utils.AppInfoParser - App info kafka.consumer for consumer-kafka-supervisor-knonnadg-1 unregistered
2022-01-05T11:53:20,989 INFO [task-runner-0-priority-0] org.apache.druid.curator.announcement.Announcer - Unannouncing [/druid/internal-discovery/PEON/10.85.6.250:8102]
2022-01-05T11:53:21,002 INFO [task-runner-0-priority-0] org.apache.druid.curator.discovery.CuratorDruidNodeAnnouncer - Unannounced self [{"druidNode":{"service":"druid/middleManager","host":"10.85.6.250","bindOnHost":false,"plaintextPort":8102,"port":-1,"tlsPort":-1,"enablePlaintextPort":true,"enableTlsPort":false},"nodeType":"peon","services":{"dataNodeService":{"type":"dataNodeService","tier":"_default_tier","maxSize":0,"type":"indexer-executor","priority":0},"lookupNodeService":{"type":"lookupNodeService","lookupTier":"__default"}}}].
2022-01-05T11:53:21,002 INFO [task-runner-0-priority-0] org.apache.druid.curator.announcement.Announcer - Unannouncing [/druid/announcements/10.85.6.250:8102]
2022-01-05T11:53:21,014 INFO [task-runner-0-priority-0] org.apache.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: {
"id" : "index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao",
"status" : "SUCCESS",
"duration" : 181183,
"errorMsg" : null,
"location" : {
"host" : null,
"port" : -1,
"tlsPort" : -1
}
}
2022-01-05T11:53:21,021 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Stopping lifecycle [module] stage [ANNOUNCEMENTS]
2022-01-05T11:53:21,026 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Stopping lifecycle [module] stage [SERVER]
2022-01-05T11:53:21,031 INFO [main] org.eclipse.jetty.server.AbstractConnector - Stopped ServerConnector@3f1d6a13{HTTP/1.1, (http/1.1)}{0.0.0.0:8102}
2022-01-05T11:53:21,031 INFO [main] org.eclipse.jetty.server.session - node0 Stopped scavenging
2022-01-05T11:53:21,032 INFO [main] org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.s.ServletContextHandler@38826984{/,null,STOPPED}
2022-01-05T11:53:21,036 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Stopping lifecycle [module] stage [NORMAL]
2022-01-05T11:53:21,036 INFO [main] org.apache.druid.server.coordination.ZkCoordinator - Stopping ZkCoordinator for [DruidServerMetadata{name='10.85.6.250:8102', hostAndPort='10.85.6.250:8102', hostAndTlsPort='null', maxSize=0, tier='_default_tier', type=indexer-executor, priority=0}]
2022-01-05T11:53:21,036 INFO [main] org.apache.druid.server.coordination.SegmentLoadDropHandler - Stopping...
2022-01-05T11:53:21,036 INFO [main] org.apache.druid.server.coordination.SegmentLoadDropHandler - Stopped.
2022-01-05T11:53:21,039 INFO [main] org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner - Starting graceful shutdown of task[index_kafka_DATASOURCE_NAME_11fc40b96a318a6_pagmfbao].
2022-01-05T11:53:21,039 INFO [main] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Stopping forcefully (status: [PUBLISHING])
2022-01-05T11:53:21,053 INFO [LookupExtractorFactoryContainerProvider-MainThread] org.apache.druid.query.lookup.LookupReferencesManager - Lookup Management loop exited. Lookup notices are not handled anymore.
2022-01-05T11:53:21,054 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [definitionkey_to_fachverfahren].
2022-01-05T11:53:21,054 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [batchstate_to_detailstate].
2022-01-05T11:53:21,054 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [batchstate_to_description].
2022-01-05T11:53:21,054 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [batchstate_to_overviewstate].
2022-01-05T11:53:21,054 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - Closed lookup [definitionkey_to_programmname].
2022-01-05T11:53:21,054 INFO [main] org.apache.druid.security.basic.authorization.db.cache.CoordinatorPollingBasicAuthorizerCacheManager - CoordinatorPollingBasicAuthorizerCacheManager is stopping.
2022-01-05T11:53:21,054 INFO [main] org.apache.druid.security.basic.authorization.db.cache.CoordinatorPollingBasicAuthorizerCacheManager - CoordinatorPollingBasicAuthorizerCacheManager is stopped.
2022-01-05T11:53:21,054 INFO [main] org.apache.druid.security.basic.authentication.db.cache.CoordinatorPollingBasicAuthenticatorCacheManager - CoordinatorPollingBasicAuthenticatorCacheManager is stopping.
2022-01-05T11:53:21,054 INFO [main] org.apache.druid.security.basic.authentication.db.cache.CoordinatorPollingBasicAuthenticatorCacheManager - CoordinatorPollingBasicAuthenticatorCacheManager is stopped.
2022-01-05T11:53:21,065 INFO [Curator-Framework-0] org.apache.curator.framework.imps.CuratorFrameworkImpl - backgroundOperationsLoop exiting
2022-01-05T11:53:21,173 INFO [main] org.apache.zookeeper.ZooKeeper - Session: 0x20006c33aae8561 closed
2022-01-05T11:53:21,173 INFO [main-EventThread] org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x20006c33aae8561
2022-01-05T11:53:21,202 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$CloseableHandler - Closing object[org.asynchttpclient.DefaultAsyncHttpClient@49cceb19]
2022-01-05T11:53:21,203 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Stopping lifecycle [module] stage [INIT]
Finished peon task
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org