You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by 刘波 <10...@qq.com> on 2021/01/08 09:07:00 UTC
Problematic thread 0x3a Query be55b109-e1f2-32ff-f8fa-7e5b3948e42 f-58, query id: be55b109-e1f2-32ff-f8fa-7e5b3948e42f
dear users,
when I use in clause,I got this error.
but when not use in cluase, the query is ok.
SELECT
COUNT(*) frequency, COUNT(DISTINCT sponsor) visitor
FROM
mt_trace_behavior
WHERE
staffid = 116
AND subtype =106
AND day_start >= '2021-01-08'
AND day_start <= '2021-01-08'
any ideas? thanks
========================================================================= 11501
2021-01-08 08:59:47,951 TRACE [streaming_job_submitter-thread-1] coordinate.BuildJobSubmitter:530 : Dump JobCheckList:
2021-01-08 08:59:47,951 INFO [streaming_job_submitter-thread-1] coordinate.BuildJobSubmitter:157 : Force traverse all cubes periodically.
2021-01-08 08:59:50,349 INFO [FetcherRunner 217643739-43] threadpool.DefaultFetcherRunner:115 : Job Fetcher: 0 should running, 0 actual running,
0 stopped, 0 ready, 29 already succeed, 2 error, 0 discarded, 0 others
2021-01-08 08:59:53,085 INFO [BadQueryDetector] service.BadQueryDetector:148 : Detect bad query.
2021-01-08 08:59:53,085 INFO [BadQueryDetector] service.BadQueryDetector:204 : Slow query has been running 135.186 seconds (project:mm_trace
_2, thread: 0x3a, user:ADMIN, query id:be55b109-e1f2-32ff-f8fa-7e5b3948e42f) --
SELECT
COUNT(*) frequency, COUNT(DISTINCT sponsor) visitor
FROM
mt_trace_behavior
WHERE
staffid = 116
AND subtype IN (106 , 116, 113)
AND day_start >= '2021-01-08'
AND day_start <= '2021-01-08'
2021-01-08 08:59:53,087 DEBUG [BadQueryDetector] badquery.BadQueryHistoryManager:65 : Loaded 32 Bad Query(s)
2021-01-08 08:59:53,089 INFO [BadQueryDetector] service.BadQueryDetector:192 : Problematic thread 0x3a Query be55b109-e1f2-32ff-f8fa-7e5b3948e42
f-58, query id: be55b109-e1f2-32ff-f8fa-7e5b3948e42f
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
at org.apache.kylin.storage.hbase.cube.v2.ExpectedSizeIterator.next(ExpectedSizeIterator.java:67)
at org.apache.kylin.storage.hbase.cube.v2.ExpectedSizeIterator.next(ExpectedSizeIterator.java:33)
at org.apache.kylin.shaded.com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47)
at org.apache.kylin.storage.gtrecord.SortMergedPartitionResultIterator.hasNext(SortMergedPartitionResultIterator.java:70)
at org.apache.kylin.shaded.com.google.common.collect.Iterators$PeekingImpl.hasNext(Iterators.java:1127)
at org.apache.kylin.gridtable.GTStreamAggregateScanner$AbstractStreamMergeIterator.hasNext(GTStreamAggregateScanner.java:80)
2021-01-08 09:00:01,799 DEBUG [http-bio-7070-exec-8] kafka.KafkaSource:90 : Last segment exists, continue from last segment 423_432's end positio
n: {0=432}
2021-01-08 09:00:01,804 INFO [http-bio-7070-exec-8] kafka.KafkaSource:114 : Get 1 partitions for topic mb-trace-topic
2021-01-08 09:00:01,805 DEBUG [http-bio-7070-exec-8] kafka.KafkaSource:125 : Seek end offsets from topic mb-trace-topic
2021-01-08 09:00:01,811 DEBUG [http-bio-7070-exec-8] kafka.KafkaSource:127 : The end offsets are {0=432}
2021-01-08 09:00:01,811 ERROR [http-bio-7070-exec-8] controller.CubeController:405 : No new message comes, startOffset = endOffset:432
java.lang.IllegalArgumentException: No new message comes, startOffset = endOffset:432
at org.apache.kylin.source.kafka.KafkaSource.enrichSourcePartitionBeforeBuild(KafkaSource.java:157)
at org.apache.kylin.rest.service.JobService.submitJobInternal(JobService.java:256)
at org.apache.kylin.rest.service.JobService.submitJob(JobService.java:226)
at org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:402)
回复: Problematic thread 0x3a Query be55b109-e1f2-32ff-f8fa-7e5b3948e42 f-58, query id: be55b109-e1f2-32ff-f8fa-7e5b3948e42f
Posted by 刘波 <10...@qq.com>.
when use two (116,113) the query is ok,
but when use three (106 , 116, 113) the query is error. it's strenge..
below is ok:
SELECT
COUNT(*) frequency, COUNT(DISTINCT sponsor) visitor
FROM
mt_trace_behavior
WHERE
staffid = 116
AND subtype IN (116,113)
AND day_start >= '2021-01-08'
AND day_start <= '2021-01-08'
------------------ 原始邮件 ------------------
发件人: "user" <1091643978@qq.com>;
发送时间: 2021年1月8日(星期五) 下午5:07
收件人: "user"<user@kylin.apache.org>;
主题: Problematic thread 0x3a Query be55b109-e1f2-32ff-f8fa-7e5b3948e42 f-58, query id: be55b109-e1f2-32ff-f8fa-7e5b3948e42f
dear users,
when I use in clause,I got this error.
but when not use in cluase, the query is ok.
SELECT
COUNT(*) frequency, COUNT(DISTINCT sponsor) visitor
FROM
mt_trace_behavior
WHERE
staffid = 116
AND subtype =106
AND day_start >= '2021-01-08'
AND day_start <= '2021-01-08'
any ideas? thanks
========================================================================= 11501
2021-01-08 08:59:47,951 TRACE [streaming_job_submitter-thread-1] coordinate.BuildJobSubmitter:530 : Dump JobCheckList:
2021-01-08 08:59:47,951 INFO [streaming_job_submitter-thread-1] coordinate.BuildJobSubmitter:157 : Force traverse all cubes periodically.
2021-01-08 08:59:50,349 INFO [FetcherRunner 217643739-43] threadpool.DefaultFetcherRunner:115 : Job Fetcher: 0 should running, 0 actual running,
0 stopped, 0 ready, 29 already succeed, 2 error, 0 discarded, 0 others
2021-01-08 08:59:53,085 INFO [BadQueryDetector] service.BadQueryDetector:148 : Detect bad query.
2021-01-08 08:59:53,085 INFO [BadQueryDetector] service.BadQueryDetector:204 : Slow query has been running 135.186 seconds (project:mm_trace
_2, thread: 0x3a, user:ADMIN, query id:be55b109-e1f2-32ff-f8fa-7e5b3948e42f) --
SELECT
COUNT(*) frequency, COUNT(DISTINCT sponsor) visitor
FROM
mt_trace_behavior
WHERE
staffid = 116
AND subtype IN (106 , 116, 113)
AND day_start >= '2021-01-08'
AND day_start <= '2021-01-08'
2021-01-08 08:59:53,087 DEBUG [BadQueryDetector] badquery.BadQueryHistoryManager:65 : Loaded 32 Bad Query(s)
2021-01-08 08:59:53,089 INFO [BadQueryDetector] service.BadQueryDetector:192 : Problematic thread 0x3a Query be55b109-e1f2-32ff-f8fa-7e5b3948e42
f-58, query id: be55b109-e1f2-32ff-f8fa-7e5b3948e42f
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
at org.apache.kylin.storage.hbase.cube.v2.ExpectedSizeIterator.next(ExpectedSizeIterator.java:67)
at org.apache.kylin.storage.hbase.cube.v2.ExpectedSizeIterator.next(ExpectedSizeIterator.java:33)
at org.apache.kylin.shaded.com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47)
at org.apache.kylin.storage.gtrecord.SortMergedPartitionResultIterator.hasNext(SortMergedPartitionResultIterator.java:70)
at org.apache.kylin.shaded.com.google.common.collect.Iterators$PeekingImpl.hasNext(Iterators.java:1127)
at org.apache.kylin.gridtable.GTStreamAggregateScanner$AbstractStreamMergeIterator.hasNext(GTStreamAggregateScanner.java:80)
2021-01-08 09:00:01,799 DEBUG [http-bio-7070-exec-8] kafka.KafkaSource:90 : Last segment exists, continue from last segment 423_432's end positio
n: {0=432}
2021-01-08 09:00:01,804 INFO [http-bio-7070-exec-8] kafka.KafkaSource:114 : Get 1 partitions for topic mb-trace-topic
2021-01-08 09:00:01,805 DEBUG [http-bio-7070-exec-8] kafka.KafkaSource:125 : Seek end offsets from topic mb-trace-topic
2021-01-08 09:00:01,811 DEBUG [http-bio-7070-exec-8] kafka.KafkaSource:127 : The end offsets are {0=432}
2021-01-08 09:00:01,811 ERROR [http-bio-7070-exec-8] controller.CubeController:405 : No new message comes, startOffset = endOffset:432
java.lang.IllegalArgumentException: No new message comes, startOffset = endOffset:432
at org.apache.kylin.source.kafka.KafkaSource.enrichSourcePartitionBeforeBuild(KafkaSource.java:157)
at org.apache.kylin.rest.service.JobService.submitJobInternal(JobService.java:256)
at org.apache.kylin.rest.service.JobService.submitJob(JobService.java:226)
at org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:402)