You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2019/07/05 09:26:10 UTC
[GitHub] [incubator-druid] quenlang opened a new issue #8032: "Possible
corruption: Memory capacity too small: 47136 < 12845088" exception occur in
0.14.x but not in 0.13.x
quenlang opened a new issue #8032: "Possible corruption: Memory capacity too small: 47136 < 12845088" exception occur in 0.14.x but not in 0.13.x
URL: https://github.com/apache/incubator-druid/issues/8032
Hello
An exception "Possible corruption: Memory capacity too small: 47136 < 12845088" occurs in the historical log when I performed a query with the Quantiles Sketch in druid 0.14.2 like this
```
...
"aggregations": [{
"type": "quantilesDoublesSketch",
"name": "resp_time_his",
"fieldName": "resp_time_his",
"k": 256
}],
"postAggregations": [{
"type": "quantilesDoublesSketchToQuantiles",
"name": "respTimeHis",
"field": {
"type": "fieldAccess",
"name": null,
"fieldName": "resp_time_his"
},
"fractions": [0.25, 0.50, 0.75, 0.99]
}],
...
```
Then I changed ```k=256``` to ```k=128```, it worked.
Also, I downgraded to druid 0.13.0 and set ```k=256```, it worked.
The historical jvm configurations:
```
-server
-Xms1g
-Xmx3g
-XX:MaxDirectMemorySize=4g
-XX:NewRatio=4
-XX:+UseConcMarkSweepGC
-Duser.timezone=UTC
-Dfile.encoding=UTF-8
-Djava.io.tmpdir=var/tmp
-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager
```
The historical runtime configurations:
```
druid.service=druid/historical
druid.port=8083
druid.server.http.numThreads=25
druid.processing.buffer.sizeBytes=200000000
druid.processing.numThreads=4
druid.processing.numMergeBuffers=1
druid.segmentCache.locations=[{"path":"var/druid/segment-cache","maxSize":130000000000}]
druid.server.maxSize=130000000000
```
And the dataset which I queryed is very small, I have no idea of this exception.
@himanshug Can you give me some advices? Thank you !
Have a nice day!
The detail error messag:
```
2019-07-05T05:58:57,982 ERROR [qtp700171153-70[groupBy_[SVR_ACTION_DATA_MIN]_d149d0bf-6bb7-4569-bbbc-43ebb5567022]] org.apache.druid.server.QueryResource - Exception handling request: {class=org.apache.druid.server.QueryResource, exceptionType=class com.yahoo.sketches.SketchesArgumentException, exceptionMessage=Possible corruption: Memory capacity too small: 47136 < 12845088, exception=com.yahoo.sketches.SketchesArgumentException: Possible corruption: Memory capacity too small: 47136 < 12845088, query=GroupByQuery{dataSource='SVR_ACTION_DATA_MIN', querySegmentSpec=MultipleSpecificSegmentSpec{descriptors=[SegmentDescriptor{interval=2019-07-04T08:05:00.000Z/2019-07-04T08:35:00.000Z, version='2019-07-04T08:01:31.683Z', partitionNumber=0}, SegmentDescriptor{interval=2019-07-04T08:05:00.000Z/2019-07-04T08:35:00.000Z, version='2019-07-04T08:01:31.683Z', partitionNumber=1}]}, virtualColumns=[], limitSpec=DefaultLimitSpec{columns='[]', limit=2147483647}, dimFilter=(biz_system_id = 1150 && data_type = APP), granularity=AllGranularity, dimensions=[DefaultDimensionSpec{dimension='biz_system_id', outputName='bizSystemId', outputType='STRING'}], aggregatorSpecs=[DoublesSketchAggregatorFactory{name=resp_time_his, fieldName=resp_time_his, k=256}], postAggregatorSpecs=[DoublesSketchToQuantilesPostAggregator{name='respTimeHis', field=FieldAccessPostAggregator{name='null', fieldName='resp_time_his'}, fractions=[0.2199999988079071, 0.550000011920929, 0.9800000190734863, 0.9900000095367432]}], havingSpec=null}, peer=10.128.1.108}
com.yahoo.sketches.SketchesArgumentException: Possible corruption: Memory capacity too small: 47136 < 12845088
at com.yahoo.sketches.quantiles.DirectUpdateDoublesSketchR.checkDirectMemCapacity(DirectUpdateDoublesSketchR.java:197) ~[?:?]
at com.yahoo.sketches.quantiles.DirectUpdateDoublesSketch.wrapInstance(DirectUpdateDoublesSketch.java:108) ~[?:?]
at com.yahoo.sketches.quantiles.DoublesUnionImpl.updateLogic(DoublesUnionImpl.java:194) ~[?:?]
at com.yahoo.sketches.quantiles.DoublesUnionImpl.update(DoublesUnionImpl.java:104) ~[?:?]
at org.apache.druid.query.aggregation.datasketches.quantiles.DoublesSketchMergeBufferAggregator.aggregate(DoublesSketchMergeBufferAggregator.java:70) ~[?:?]
at org.apache.druid.query.groupby.epinephelinae.AbstractBufferHashGrouper.aggregate(AbstractBufferHashGrouper.java:165) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.query.groupby.epinephelinae.SpillingGrouper.aggregate(SpillingGrouper.java:167) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.query.groupby.epinephelinae.ConcurrentGrouper.aggregate(ConcurrentGrouper.java:267) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.query.groupby.epinephelinae.Grouper.aggregate(Grouper.java:82) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.query.groupby.epinephelinae.RowBasedGrouperHelper$1.accumulate(RowBasedGrouperHelper.java:270) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.query.groupby.epinephelinae.RowBasedGrouperHelper$1.accumulate(RowBasedGrouperHelper.java:247) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.MappingAccumulator.accumulate(MappingAccumulator.java:40) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.BaseSequence.accumulate(BaseSequence.java:45) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.ConcatSequence.lambda$accumulate$0(ConcatSequence.java:43) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.MappingAccumulator.accumulate(MappingAccumulator.java:40) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.FilteringAccumulator.accumulate(FilteringAccumulator.java:41) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.MappingAccumulator.accumulate(MappingAccumulator.java:40) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.BaseSequence.accumulate(BaseSequence.java:45) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.MappedSequence.accumulate(MappedSequence.java:43) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.FilteredSequence.accumulate(FilteredSequence.java:45) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.MappedSequence.accumulate(MappedSequence.java:43) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.ConcatSequence.accumulate(ConcatSequence.java:43) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.LazySequence.accumulate(LazySequence.java:40) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.MappedSequence.accumulate(MappedSequence.java:43) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.LazySequence.accumulate(LazySequence.java:40) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.query.spec.SpecificSegmentQueryRunner$1.accumulate(SpecificSegmentQueryRunner.java:82) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.query.spec.SpecificSegmentQueryRunner.doNamed(SpecificSegmentQueryRunner.java:168) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.query.spec.SpecificSegmentQueryRunner.access$100(SpecificSegmentQueryRunner.java:45) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.query.spec.SpecificSegmentQueryRunner$2.wrap(SpecificSegmentQueryRunner.java:148) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.query.CPUTimeMetricQueryRunner$1.wrap(CPUTimeMetricQueryRunner.java:74) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.query.groupby.epinephelinae.GroupByMergingQueryRunnerV2$1$1$1.call(GroupByMergingQueryRunnerV2.java:244) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.query.groupby.epinephelinae.GroupByMergingQueryRunnerV2$1$1$1.call(GroupByMergingQueryRunnerV2.java:232) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_60]
at org.apache.druid.query.PrioritizedListenableFutureTask.run(PrioritizedExecutorService.java:247) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_60]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_60]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org