You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2019/02/21 01:38:11 UTC
[GitHub] clintropolis commented on a change in pull request #7116: segment
metadata fallback analysis if no bitmaps
clintropolis commented on a change in pull request #7116: segment metadata fallback analysis if no bitmaps
URL: https://github.com/apache/incubator-druid/pull/7116#discussion_r258749692
##########
File path: processing/src/main/java/org/apache/druid/query/metadata/SegmentAnalyzer.java
##########
@@ -198,26 +199,42 @@ private ColumnAnalysis analyzeStringColumn(
Comparable min = null;
Comparable max = null;
-
- if (!capabilities.hasBitmapIndexes()) {
- return ColumnAnalysis.error("string_no_bitmap");
- }
-
- final BitmapIndex bitmapIndex = columnHolder.getBitmapIndex();
- final int cardinality = bitmapIndex.getCardinality();
-
- if (analyzingSize()) {
- for (int i = 0; i < cardinality; ++i) {
- String value = bitmapIndex.getValue(i);
- if (value != null) {
- size += StringUtils.estimatedBinaryLengthAsUTF8(value) * bitmapIndex.getBitmap(bitmapIndex.getIndex(value)).size();
+ int cardinality = 0;
+ if (capabilities.hasBitmapIndexes()) {
+ final BitmapIndex bitmapIndex = columnHolder.getBitmapIndex();
+ cardinality = bitmapIndex.getCardinality();
+
+ if (analyzingSize()) {
+ for (int i = 0; i < cardinality; ++i) {
+ String value = bitmapIndex.getValue(i);
+ if (value != null) {
+ size += StringUtils.estimatedBinaryLengthAsUTF8(value) * bitmapIndex.getBitmap(bitmapIndex.getIndex(value))
+ .size();
+ }
}
}
- }
- if (analyzingMinMax() && cardinality > 0) {
- min = NullHandling.nullToEmptyIfNeeded(bitmapIndex.getValue(0));
- max = NullHandling.nullToEmptyIfNeeded(bitmapIndex.getValue(cardinality - 1));
+ if (analyzingMinMax() && cardinality > 0) {
+ min = NullHandling.nullToEmptyIfNeeded(bitmapIndex.getValue(0));
+ max = NullHandling.nullToEmptyIfNeeded(bitmapIndex.getValue(cardinality - 1));
+ }
+ } else if (capabilities.isDictionaryEncoded()) {
+ // fallback if no bitmap index
Review comment:
Agree. I think it would be interesting to have size be representative of "estimated size in memory" for incremental index, or size in bytes of column in the segment file that is mapped, but what it's computing here seems like basically nonsense for all column types.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org