You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/02/11 05:46:03 UTC
[GitHub] [incubator-doris] spaces-X opened a new issue #8017: [Bug] Be memory leak when insert data with bitmap type
spaces-X opened a new issue #8017:
URL: https://github.com/apache/incubator-doris/issues/8017
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
### Version
tested in `pre-0.15.0-5fc0a9f` and `trunk-a1d1bd8 (0.15)`
### What's Wrong?
DCHECK failed in Memtracker destructor:
`consumption() == 0`
### What You Expected?
DCHECK pass
### How to Reproduce?
CREATE TABLE test_bitmap
(
k1 int(11) NULL comment "date",
k2 INT NULL comment "k2 int",
v1 bitmap bitmap_union,
v2 INT SUM
)
AGGREGATE KEY(k1, k2)
PARTITION BY RANGE (k1)
(
PARTITION p1 VALUES [("20210101"), ("20210102")),
PARTITION p2 VALUES [("20210102"), ("20210103"))
)
DISTRIBUTED BY HASH(k2) BUCKETS 3
PROPERTIES (
"replication_num" = "1",
"in_memory" = "false",
"storage_format" = "V2"
);
insert into test_bitmap values (20210101, 1, to_bitmap(10),50);
### Anything Else?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] spaces-X edited a comment on issue #8017: [Bug] Be memory leak when insert data with bitmap type
Posted by GitBox <gi...@apache.org>.
spaces-X edited a comment on issue #8017:
URL: https://github.com/apache/incubator-doris/issues/8017#issuecomment-1035942556
DCHECK failure in memtracker destructor is caused by memtrack consumed the BitmapValue without releasing later in AggregateFuncTraits::init.
As follows:
```
template <>
struct AggregateFuncTraits<OLAP_FIELD_AGGREGATION_BITMAP_UNION, OLAP_FIELD_TYPE_OBJECT> {
static void init(RowCursorCell* dst, const char* src, bool src_null, MemPool* mem_pool,
ObjectPool* agg_pool) {
DCHECK_EQ(src_null, false);
dst->set_not_null();
auto* src_slice = reinterpret_cast<const Slice*>(src);
auto* dst_slice = reinterpret_cast<Slice*>(dst->mutable_cell_ptr());
// we use zero size represent this slice is a agg object
dst_slice->size = 0;
auto bitmap = new BitmapValue(src_slice->data);
// here consumed size of (BitmapValue) without releasing later.
mem_pool->mem_tracker()->Consume(sizeof(BitmapValue));
dst_slice->data = (char*)bitmap;
agg_pool->add(bitmap);
}
...
}
```
Any suggestiones here or we can just ignore this DCHECK ?
@morningman @liutang123
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] spaces-X commented on issue #8017: [Bug] Be memory leak when insert data with bitmap type
Posted by GitBox <gi...@apache.org>.
spaces-X commented on issue #8017:
URL: https://github.com/apache/incubator-doris/issues/8017#issuecomment-1035942556
DCHECK failure in memtracker destructor is caused by memtrack consumed the BitmapValue without releasing later in AggregateFuncTraits::init.
As follows:
```
template <>
struct AggregateFuncTraits<OLAP_FIELD_AGGREGATION_BITMAP_UNION, OLAP_FIELD_TYPE_OBJECT> {
static void init(RowCursorCell* dst, const char* src, bool src_null, MemPool* mem_pool,
ObjectPool* agg_pool) {
DCHECK_EQ(src_null, false);
dst->set_not_null();
auto* src_slice = reinterpret_cast<const Slice*>(src);
auto* dst_slice = reinterpret_cast<Slice*>(dst->mutable_cell_ptr());
// we use zero size represent this slice is a agg object
dst_slice->size = 0;
auto bitmap = new BitmapValue(src_slice->data);
// here consumed size of (BitmapValue) without releasing later.
mem_pool->mem_tracker()->Consume(sizeof(BitmapValue));
dst_slice->data = (char*)bitmap;
agg_pool->add(bitmap);
}
...
}
```
Any suggestiones here or we can just skip this DCHECK ?
@morningman @liutang123
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org