You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2019/07/31 05:11:48 UTC
[GitHub] [incubator-druid] quenlang commented on issue #8095: Incorrect
quantiles result produced by quantilesDoublesSketch
quenlang commented on issue #8095: Incorrect quantiles result produced by quantilesDoublesSketch
URL: https://github.com/apache/incubator-druid/issues/8095#issuecomment-516698275
@AlexanderSaydakov
Hi, I merged 7666 into 0.14.2 and build from the source. Then an exception occurred at data ingestion.
It seems a ```groupBy``` query performed at realtime peon task which caused this exception.
```
2019-07-31T04:09:25,691 INFO [task-runner-0-priority-0] org.apache.druid.query.aggregation.datasketches.quantiles.DoublesSketchOperations - deser sketch: AgMIAIAAAAABAAAAAAAAAAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==
2019-07-31T04:09:25,691 INFO [task-runner-0-priority-0] org.apache.druid.query.aggregation.datasketches.quantiles.DoublesSketchOperations - deser sketch(bin): AgMIAIAAAAABAAAAAAAAAAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==
2019-07-31T04:09:25,691 INFO [task-runner-0-priority-0] org.apache.druid.query.aggregation.datasketches.quantiles.DoublesSketchMergeAggregator - aggregate sketch by merge aggregator: AgMIGoAAAAABAAAAAAAAAAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQA==
2019-07-31T04:09:25,738 INFO [task-runner-0-priority-0] org.apache.druid.query.aggregation.datasketches.quantiles.DoublesSketchOperations - deser sketch: AgMIGoAAAAASAAAAAAAAAAAAAAAAAAAAAAAAAACAQEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAACAQEA=
2019-07-31T04:09:25,738 INFO [task-runner-0-priority-0] org.apache.druid.query.aggregation.datasketches.quantiles.DoublesSketchOperations - deser sketch(bin): AgMIGoAAAAASAAAAAAAAAAAAAAAAAAAAAAAAAACAQEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAACAQEA=
2019-07-31T04:09:25,738 INFO [task-runner-0-priority-0] org.apache.druid.query.aggregation.datasketches.quantiles.DoublesSketchMergeAggregator - aggregate sketch by merge aggregator: AgMIGoAAAAASAAAAAAAAAAAAAAAAAAAAAAAAAACAQEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAACAQEA=
2019-07-31T04:09:25,741 INFO [task-runner-0-priority-0] org.apache.druid.query.aggregation.datasketches.quantiles.DoublesSketchOperations - deser sketch: AgMIAIAAAAASAAAAAAAAAAAAAAAAAAAAAAAAAACAQEAAAAAAAAAAAAAAAAAAAPA/AAAAAAAA8D8AAAAAAAAAAAAAAAAAAPA/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADwPwAAAAAAAAAAAAAAAAAA8D8AAAAAAIBAQAAAAAAAAAAAAAAAAAAA8D8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
2019-07-31T04:09:25,741 INFO [task-runner-0-priority-0] org.apache.druid.query.aggregation.datasketches.quantiles.DoublesSketchOperations - deser sketch(bin): AgMIAIAAAAASAAAAAAAAAAAAAAAAAAAAAAAAAACAQEAAAAAAAAAAAAAAAAAAAPA/AAAAAAAA8D8AAAAAAAAAAAAAAAAAAPA/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADwPwAAAAAAAAAAAAAAAAAA8D8AAAAAAIBAQAAAAAAAAAAAAAAAAAAA8D8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
2019-07-31T04:09:25,741 INFO [task-runner-0-priority-0] org.apache.druid.query.aggregation.datasketches.quantiles.DoublesSketchMergeAggregator - aggregate sketch by merge aggregator: AgMIGoAAAAASAAAAAAAAAAAAAAAAAAAAAAAAAACAQEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAACAQEA=
2019-07-31T04:09:25,744 INFO [groupBy_SVR_ACTION_DATA_MIN_[2019-07-31T03:39:00.000Z/2019-07-31T04:09:00.000Z]] org.apache.druid.query.aggregation.datasketches.quantiles.DoublesSketchMergeBufferAggregator - aggregate sketch by mergebuf aggregator: AgMIGoAAAACZAQAAAAAAAAAAAAAAAAAAAAAAAIBBx0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAHEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIkAAAAAAAAAiQAAAAAAAAChAAAAAAAAAKkAAAAAAAAAwQAAAAAAAADRAAAAAAAAAOkAAAAAAAAA9QAAAAAAAAD5AAAAAAAAAP0AAAAAAANCHQAAAAACAJsVAAAAAAIBBx0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIkAAAAAAAAAkQAAAAAAAAChAAAAAAAAALkAAAAAAAAA4QAAAAAAAADtAAAAAAAAAPkAAAAAAAIBAQAAAAAAAgEFAAAAAAACAQkAAAAAAAIBDQAAAAAAAgENAAAAAAACASUAAAAAAAABQQAAAAAAAgFJAAAAAAABAXUAAAAAAAMBkQAAAAAAAWKFA
2019-07-31T04:09:25,744 INFO [groupBy_SVR_ACTION_DATA_MIN_[2019-07-31T03:39:00.000Z/2019-07-31T04:09:00.000Z]] org.apache.druid.query.aggregation.datasketches.quantiles.DoublesSketchMergeBufferAggregator - aggregate sketch by mergebuf aggregator: AgMIGgABAAAHAAAAAAAAAAAAAAAAAAAAAAAAAAAAHEAAAAAAAAAAAAAAAAAAAPA/AAAAAAAA8D8AAAAAAAAAQAAAAAAAAAhAAAAAAAAAGEAAAAAAAAAcQA==
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f3aadbfe3e1, pid=10710, tid=139879948633856
#
# JRE version: Java(TM) SE Runtime Environment (8.0_60-b27) (build 1.8.0_60-b27)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.60-b23 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# J 11067 C2 com.tingyun.com.yahoo.memory.NonNativeWritableMemoryImpl.getDouble(J)D (30 bytes) @ 0x00007f3aadbfe3e1 [0x00007f3aadbfe3a0+0x41]
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /data/tingyun/druid-0.12.3/hs_err_pid10710.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#
```
Releated information in ```/data/tingyun/druid-0.12.3/hs_err_pid10710.log```:
```
Register to memory mapping:
RAX=0x000000010086ac78 is pointing into metadata
RBX={method} {0x00007f38571e3b68} 'getDouble' '(J)D' in 'com/tingyun/com/yahoo/memory/NonNativeWritableMemoryImpl'
RCX=0x0000000000000080 is an unknown value
RDX=0x00007f38f00dea34 is an unknown value
RSP=0x00007f3856a61e20 is pointing into the stack for thread: 0x00007f3880010000
RBP=0x0000000000a80048 is an unknown value
RSI=0x000000009b3e8a20 is an oop
com.tingyun.com.yahoo.memory.BBNonNativeWritableMemoryImpl
- klass: 'com/tingyun/com/yahoo/memory/BBNonNativeWritableMemoryImpl'
RDI=0x0000000000000000 is an unknown value
R8 =0x0000000000000100 is an unknown value
R9 =0x000000009b3fabd8 is an oop
com.tingyun.com.yahoo.sketches.quantiles.DirectDoublesSketchAccessor
- klass: 'com/tingyun/com/yahoo/sketches/quantiles/DirectDoublesSketchAccessor'
R10=0x000000002010d58f is an unknown value
R11=0x0000000000000000 is an unknown value
R12=0x0000000000000000 is an unknown value
R13=0x00007f3856a61ed8 is pointing into the stack for thread: 0x00007f3880010000
R14=0x00007f3856a62020 is pointing into the stack for thread: 0x00007f3880010000
R15=0x00007f3880010000 is a thread
...
...
Stack: [0x00007f3856964000,0x00007f3856a65000], sp=0x00007f3856a61e20, free space=1015k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
J 11067 C2 com.tingyun.com.yahoo.memory.NonNativeWritableMemoryImpl.getDouble(J)D (30 bytes) @ 0x00007f3aadbfe3e1 [0x00007f3aadbfe3a0+0x41]
J 9754 C2 com.tingyun.com.yahoo.sketches.quantiles.DirectDoublesSketchAccessor.get(I)D (77 bytes) @ 0x00007f3aadd90ba8 [0x00007f3aadd90b20+0x88]
J 12669 C1 com.tingyun.com.yahoo.sketches.quantiles.DoublesMergeImpl.justZipWithStride(Lcom/tingyun/com/yahoo/sketches/quantiles/DoublesBufferAccessor;Lcom/tingyun/com/yahoo/sketches/quantiles/DoublesBufferAccessor;II)V (48 bytes) @ 0x00007f3aae91d6cc [0x00007f3aae91d500+0x1cc]
j com.tingyun.com.yahoo.sketches.quantiles.DoublesMergeImpl.downSamplingMergeInto(Lcom/tingyun/com/yahoo/sketches/quantiles/DoublesSketch;Lcom/tingyun/com/yahoo/sketches/quantiles/UpdateDoublesSketch;)V+198
J 13391 C2 com.tingyun.com.yahoo.sketches.quantiles.DoublesUnionImpl.updateLogic(ILcom/tingyun/com/yahoo/sketches/quantiles/UpdateDoublesSketch;Lcom/tingyun/com/yahoo/sketches/quantiles/DoublesSketch;)Lcom/tingyun/com/yahoo/sketches/quantiles/UpdateDoublesSketch; (552 bytes) @ 0x00007f3aaebd302c [0x00007f3aaebd1940+0x16ec]
J 7158 C2 com.tingyun.com.yahoo.sketches.quantiles.DoublesUnionImpl.update(Lcom/tingyun/com/yahoo/sketches/quantiles/DoublesSketch;)V (17 bytes) @ 0x00007f3aadc86d64 [0x00007f3aadc86d20+0x44]
J 13468 C1 org.apache.druid.query.aggregation.datasketches.quantiles.DoublesSketchMergeBufferAggregator.aggregate(Ljava/nio/ByteBuffer;I)V (83 bytes) @ 0x00007f3aadf51a0c [0x00007f3aadf51380+0x68c]
J 13278 C2 org.apache.druid.query.groupby.epinephelinae.Grouper.aggregate(Ljava/lang/Object;)Lorg/apache/druid/query/groupby/epinephelinae/AggregateResult; (27 bytes) @ 0x00007f3aaeb60e10 [0x00007f3aaeb5ffc0+0xe50]
J 10418 C2 org.apache.druid.query.groupby.epinephelinae.GroupByQueryEngineV2$HashAggregateIterator.aggregateSingleValueDims(Lorg/apache/druid/query/groupby/epinephelinae/Grouper;)V (117 bytes) @ 0x00007f3aae2afb70 [0x00007f3aae2afa20+0x150]
J 13638 C2 org.apache.druid.query.groupby.epinephelinae.GroupByQueryEngineV2$GroupByEngineIterator.initNewDelegate()Lorg/apache/druid/query/groupby/epinephelinae/CloseableGrouperIterator; (48 bytes) @ 0x00007f3aae217c6c [0x00007f3aae215880+0x23ec]
J 10400 C2 org.apache.druid.query.groupby.epinephelinae.GroupByQueryEngineV2$GroupByEngineIterator.hasNext()Z (57 bytes) @ 0x00007f3aae273748 [0x00007f3aae273640+0x108]
J 13110 C2 org.apache.druid.java.util.common.guava.ConcatSequence$$Lambda$136.accumulate(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; (13 bytes) @ 0x00007f3aaea8987c [0x00007f3aaea89800+0x7c]
J 13399 C2 org.apache.druid.java.util.common.guava.FilteringAccumulator.accumulate(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; (27 bytes) @ 0x00007f3aad6b39e0 [0x00007f3aad6b3940+0xa0]
```
Any suggestions? Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org