You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "xinyiZzz (via GitHub)" <gi...@apache.org> on 2023/03/15 09:27:41 UTC
[GitHub] [doris] xinyiZzz opened a new pull request, #17820: [enhancement](memory) Remove MemPool and replace it with Arena
xinyiZzz opened a new pull request, #17820:
URL: https://github.com/apache/doris/pull/17820
# Proposed changes
Issue Number: close #xxx
## Problem summary
Arena can replace MemPool in most scenarios. Except for memory reuse, MemPool supports reuse of previous memory chunks after `clear`, but Arena does not.
Some comparisons between MemPool and Arena
1. Expansion
Arena is less than 128M index 2 alloc chunk; more than 128M memory, allocate 128M * n > `size`, n is equal to the minimum value that satisfies the expression;
MemPool less than 512K index 2 alloc chunk, greater than 512K memory, separately apply for a `size` length chunk
After Arena applied for a chunk larger than 128M last time, the minimum chunk applied for after that is 128M. Does this seem to be a waste of memory? MemPool is also similar. After the chunk of 512K was applied for last time, the minimum chunk of subsequent applications is 512K.
2. Alignment
MemPool defaults to 16 alignment, because memtable and other places that use int128 require 16 alignment;
Arena has no default alignment;
3. Memory reuse
Arena only supports `rollback`, which reuses the memory of the current chunk, usually the memory requested last time.
MemPool supports clear(), all chunks can be reused; or call ReturnPartialAllocation() to roll back the last requested memory; if the last chunk has no memory, search for the most free chunk for allocation
4. Realloc
Arena supports realloc contiguous memory; it also supports realloc contiguous memory from any position at the time of the last allocation. The difference between `alloc_continue` and `realloc` is:
1. Alloc_continue does not need to specify the old size, but the default old size = head->pos - range_start
2. alloc_continue supports expansion from range_start when additional_bytes is between head and pos, which is equivalent to reusing a part of memory, while realloc completely allocates a new memory
MemPool does not support realloc, but supports transferring or absorbing chunks between two MemPools
5. check mem limit
MemPool checks the mem limit, and Arena checks at the Allocator layer.
6. Support for ASAN
Arena does something extra
7. Error handling
MemPool supports returning the error message of application failure directly through `Status`, and Arena throws Exception.
## Checklist(Required)
* [ ] Does it affect the original behavior
* [ ] Has unit tests been added
* [ ] Has document been added or modified
* [ ] Does it need to update dependencies
* [ ] Is this PR support rollback (If NO, please explain WHY)
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1476978976
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1138486846
##########
be/src/olap/rowset/segment_v2/binary_dict_page.cpp:
##########
@@ -152,7 +151,7 @@ size_t BinaryDictPageBuilder::count() const {
}
uint64_t BinaryDictPageBuilder::size() const {
- return _pool.total_allocated_bytes() + _data_page_builder->size();
+ return _arena.size() + _data_page_builder->size();
Review Comment:
same as above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1136816577
##########
be/test/olap/rowset/segment_v2/bitshuffle_page_test.cpp:
##########
@@ -39,7 +38,7 @@ class BitShufflePageTest : public testing::Test {
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
std::unique_ptr<ColumnVectorBatch> cvb;
^
```
##########
be/test/olap/rowset/segment_v2/binary_prefix_page_test.cpp:
##########
@@ -75,7 +74,7 @@ class BinaryPrefixPageTest : public testing::Test {
EXPECT_EQ(slices.size(), page_decoder->count());
//check values
- MemPool pool;
+ vectorized::Arena pool;
auto type_info = get_scalar_type_info(OLAP_FIELD_TYPE_VARCHAR);
size_t size = slices.size();
std::unique_ptr<ColumnVectorBatch> cvb;
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
std::unique_ptr<ColumnVectorBatch> cvb;
^
```
##########
be/test/olap/rowset/segment_v2/binary_prefix_page_test.cpp:
##########
@@ -96,7 +95,7 @@
int n = 0;
while (true) {
//check values
- MemPool pool;
+ vectorized::Arena pool;
auto type_info = get_scalar_type_info(OLAP_FIELD_TYPE_VARCHAR);
std::unique_ptr<ColumnVectorBatch> cvb;
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
std::unique_ptr<ColumnVectorBatch> cvb;
^
```
##########
be/test/olap/rowset/segment_v2/bitshuffle_page_test.cpp:
##########
@@ -39,7 +38,7 @@
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
^
```
##########
be/test/olap/rowset/segment_v2/bitshuffle_page_test.cpp:
##########
@@ -83,7 +82,7 @@
EXPECT_TRUE(status.ok());
EXPECT_EQ(0, page_decoder.current_index());
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
std::unique_ptr<ColumnVectorBatch> cvb;
^
```
##########
be/test/olap/rowset/segment_v2/frame_of_reference_page_test.cpp:
##########
@@ -33,7 +32,7 @@
public:
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
^
```
##########
be/test/olap/rowset/segment_v2/frame_of_reference_page_test.cpp:
##########
@@ -33,7 +32,7 @@
public:
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
Review Comment:
warning: use of undeclared identifier 'get_scalar_type_info' [clang-diagnostic-error]
```cpp
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
^
```
##########
be/test/olap/rowset/segment_v2/frame_of_reference_page_test.cpp:
##########
@@ -33,7 +32,7 @@
public:
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
Review Comment:
warning: variable has incomplete type 'vectorized::Arena' [clang-diagnostic-error]
```cpp
vectorized::Arena pool;
^
```
**be/src/vec/columns/column.h:60:** forward declaration of 'doris::vectorized::Arena'
```cpp
class Arena;
^
```
##########
be/test/olap/rowset/segment_v2/frame_of_reference_page_test.cpp:
##########
@@ -64,7 +63,7 @@
EXPECT_EQ(0, for_page_decoder.current_index());
EXPECT_EQ(size, for_page_decoder.count());
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
std::unique_ptr<ColumnVectorBatch> cvb;
^
```
##########
be/test/olap/rowset/segment_v2/frame_of_reference_page_test.cpp:
##########
@@ -64,7 +63,7 @@
EXPECT_EQ(0, for_page_decoder.current_index());
EXPECT_EQ(size, for_page_decoder.count());
- MemPool pool;
+ vectorized::Arena pool;
Review Comment:
warning: variable has incomplete type 'vectorized::Arena' [clang-diagnostic-error]
```cpp
vectorized::Arena pool;
^
```
**be/src/vec/columns/column.h:60:** forward declaration of 'doris::vectorized::Arena'
```cpp
class Arena;
^
```
##########
be/test/olap/rowset/segment_v2/frame_of_reference_page_test.cpp:
##########
@@ -64,7 +63,7 @@
EXPECT_EQ(0, for_page_decoder.current_index());
EXPECT_EQ(size, for_page_decoder.count());
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(size, true, get_scalar_type_info(Type), nullptr, &cvb);
ColumnBlock block(cvb.get(), &pool);
Review Comment:
warning: unknown type name 'ColumnBlock' [clang-diagnostic-error]
```cpp
ColumnBlock block(cvb.get(), &pool);
^
```
##########
be/test/olap/rowset/segment_v2/plain_page_test.cpp:
##########
@@ -46,7 +45,7 @@ class PlainPageTest : public testing::Test {
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
std::unique_ptr<ColumnVectorBatch> cvb;
^
```
##########
be/test/olap/rowset/segment_v2/plain_page_test.cpp:
##########
@@ -84,7 +83,7 @@
EXPECT_EQ(0, page_decoder.current_index());
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
std::unique_ptr<ColumnVectorBatch> cvb;
^
```
##########
be/test/olap/rowset/segment_v2/bitshuffle_page_test.cpp:
##########
@@ -39,7 +38,7 @@
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
ColumnBlock block(cvb.get(), &pool);
Review Comment:
warning: unknown type name 'ColumnBlock' [clang-diagnostic-error]
```cpp
ColumnBlock block(cvb.get(), &pool);
^
```
##########
be/test/olap/rowset/segment_v2/rle_page_test.cpp:
##########
@@ -37,7 +36,7 @@
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
Review Comment:
warning: variable has incomplete type 'vectorized::Arena' [clang-diagnostic-error]
```cpp
vectorized::Arena pool;
^
```
**be/src/vec/columns/column.h:60:** forward declaration of 'doris::vectorized::Arena'
```cpp
class Arena;
^
```
##########
be/test/olap/rowset/segment_v2/plain_page_test.cpp:
##########
@@ -84,7 +83,7 @@
EXPECT_EQ(0, page_decoder.current_index());
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(size, true, get_scalar_type_info(Type), nullptr, &cvb);
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
ColumnVectorBatch::create(size, true, get_scalar_type_info(Type), nullptr, &cvb);
^
```
##########
be/test/olap/rowset/segment_v2/frame_of_reference_page_test.cpp:
##########
@@ -33,7 +32,7 @@
public:
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
ColumnBlock block(cvb.get(), &pool);
Review Comment:
warning: unknown type name 'ColumnBlock' [clang-diagnostic-error]
```cpp
ColumnBlock block(cvb.get(), &pool);
^
```
##########
be/test/olap/rowset/segment_v2/frame_of_reference_page_test.cpp:
##########
@@ -33,7 +32,7 @@
public:
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
std::unique_ptr<ColumnVectorBatch> cvb;
^
```
##########
be/test/olap/rowset/segment_v2/frame_of_reference_page_test.cpp:
##########
@@ -33,7 +32,7 @@ class FrameOfReferencePageTest : public testing::Test {
public:
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
Review Comment:
warning: no template named 'TypeTraits' [clang-diagnostic-error]
```cpp
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
^
```
##########
be/test/olap/rowset/segment_v2/frame_of_reference_page_test.cpp:
##########
@@ -64,7 +63,7 @@
EXPECT_EQ(0, for_page_decoder.current_index());
EXPECT_EQ(size, for_page_decoder.count());
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(size, true, get_scalar_type_info(Type), nullptr, &cvb);
Review Comment:
warning: use of undeclared identifier 'get_scalar_type_info' [clang-diagnostic-error]
```cpp
ColumnVectorBatch::create(size, true, get_scalar_type_info(Type), nullptr, &cvb);
^
```
##########
be/test/olap/rowset/segment_v2/frame_of_reference_page_test.cpp:
##########
@@ -64,7 +63,7 @@
EXPECT_EQ(0, for_page_decoder.current_index());
EXPECT_EQ(size, for_page_decoder.count());
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(size, true, get_scalar_type_info(Type), nullptr, &cvb);
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
ColumnVectorBatch::create(size, true, get_scalar_type_info(Type), nullptr, &cvb);
^
```
##########
be/test/olap/rowset/segment_v2/bitshuffle_page_test.cpp:
##########
@@ -83,7 +82,7 @@
EXPECT_TRUE(status.ok());
EXPECT_EQ(0, page_decoder.current_index());
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(size, false, get_scalar_type_info(Type), nullptr, &cvb);
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
ColumnVectorBatch::create(size, false, get_scalar_type_info(Type), nullptr, &cvb);
^
```
##########
be/test/olap/rowset/segment_v2/plain_page_test.cpp:
##########
@@ -46,7 +45,7 @@
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
^
```
##########
be/test/olap/rowset/segment_v2/plain_page_test.cpp:
##########
@@ -46,7 +45,7 @@
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
ColumnBlock block(cvb.get(), &pool);
Review Comment:
warning: unknown type name 'ColumnBlock' [clang-diagnostic-error]
```cpp
ColumnBlock block(cvb.get(), &pool);
^
```
##########
be/test/olap/rowset/segment_v2/rle_page_test.cpp:
##########
@@ -37,7 +36,7 @@ class RlePageTest : public testing::Test {
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
Review Comment:
warning: no template named 'TypeTraits' [clang-diagnostic-error]
```cpp
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1147316519
##########
be/src/vec/common/arena.h:
##########
@@ -271,9 +276,17 @@ class Arena : private boost::noncopyable {
return res;
}
+ void clear() {
+ delete head;
+ head = new Chunk(initial_size, nullptr), size_in_bytes = head->size(),
+ size_in_allocated = 0;
+ }
+
/// Size of chunks in bytes.
size_t size() const { return size_in_bytes; }
+ size_t allocated_size() const { return size_in_allocated + head->remaining(); }
Review Comment:
rename this method to: used_size() because allocated size maybe allocated but not used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1147308651
##########
be/src/vec/common/arena.h:
##########
@@ -271,9 +276,17 @@ class Arena : private boost::noncopyable {
return res;
}
+ void clear() {
Review Comment:
remove
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1136791535
##########
be/src/olap/rowset/segment_v2/bitmap_index_writer.cpp:
##########
@@ -64,7 +64,7 @@ class BitmapIndexWriterImpl : public BitmapIndexWriter {
using MemoryIndexType = typename BitmapIndexTraits<CppType>::MemoryIndexType;
explicit BitmapIndexWriterImpl(const TypeInfo* type_info)
- : _type_info(type_info), _reverted_index_size(0), _pool() {}
+ : _type_info(type_info), _reverted_index_size(0), _arena() {}
Review Comment:
warning: initializer for member '_arena' is redundant [readability-redundant-member-init]
```suggestion
: _type_info(type_info), _reverted_index_size(0), {}
```
##########
be/src/olap/rowset/segment_v2/indexed_column_writer.cpp:
##########
@@ -40,7 +40,7 @@ IndexedColumnWriter::IndexedColumnWriter(const IndexedColumnWriterOptions& optio
: _options(options),
_type_info(type_info),
_file_writer(file_writer),
- _mem_pool(),
+ _arena(),
Review Comment:
warning: initializer for member '_arena' is redundant [readability-redundant-member-init]
```suggestion
,
```
##########
be/test/olap/ordered_data_compaction_test.cpp:
##########
@@ -213,15 +213,15 @@
uint32_t num_rows = 0;
for (int i = 0; i < rowset_data.size(); ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rowset_data[i].size(); ++rid) {
uint32_t c1 = std::get<0>(rowset_data[i][rid]);
uint32_t c2 = std::get<1>(rowset_data[i][rid]);
- input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &arena);
^
```
##########
be/test/olap/ordered_data_compaction_test.cpp:
##########
@@ -213,15 +213,15 @@
uint32_t num_rows = 0;
for (int i = 0; i < rowset_data.size(); ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rowset_data[i].size(); ++rid) {
uint32_t c1 = std::get<0>(rowset_data[i][rid]);
uint32_t c2 = std::get<1>(rowset_data[i][rid]);
- input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &arena);
if (tablet_schema->keys_type() == UNIQUE_KEYS) {
uint8_t num = 0;
- input_row.set_field_content(2, reinterpret_cast<char*>(&num), &mem_pool);
+ input_row.set_field_content(2, reinterpret_cast<char*>(&num), &arena);
}
s = rowset_writer->add_row(input_row);
Review Comment:
warning: no member named 'add_row' in 'doris::RowsetWriter' [clang-diagnostic-error]
```cpp
s = rowset_writer->add_row(input_row);
^
```
##########
be/test/olap/ordered_data_compaction_test.cpp:
##########
@@ -213,15 +213,15 @@
uint32_t num_rows = 0;
for (int i = 0; i < rowset_data.size(); ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rowset_data[i].size(); ++rid) {
uint32_t c1 = std::get<0>(rowset_data[i][rid]);
uint32_t c2 = std::get<1>(rowset_data[i][rid]);
- input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &arena);
if (tablet_schema->keys_type() == UNIQUE_KEYS) {
uint8_t num = 0;
- input_row.set_field_content(2, reinterpret_cast<char*>(&num), &mem_pool);
+ input_row.set_field_content(2, reinterpret_cast<char*>(&num), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(2, reinterpret_cast<char*>(&num), &arena);
^
```
##########
be/test/olap/row_cursor_test.cpp:
##########
@@ -344,31 +343,31 @@
Slice l_char("well");
int32_t l_int = 10;
- left.set_field_content(0, reinterpret_cast<char*>(&l_char), _mem_pool.get());
- left.set_field_content(1, reinterpret_cast<char*>(&l_int), _mem_pool.get());
+ left.set_field_content(0, reinterpret_cast<char*>(&l_char), _arena.get());
+ left.set_field_content(1, reinterpret_cast<char*>(&l_int), _arena.get());
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
left.set_field_content(1, reinterpret_cast<char*>(&l_int), _arena.get());
^
```
##########
be/test/olap/row_cursor_test.cpp:
##########
@@ -344,31 +343,31 @@
Slice l_char("well");
int32_t l_int = 10;
- left.set_field_content(0, reinterpret_cast<char*>(&l_char), _mem_pool.get());
- left.set_field_content(1, reinterpret_cast<char*>(&l_int), _mem_pool.get());
+ left.set_field_content(0, reinterpret_cast<char*>(&l_char), _arena.get());
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
left.set_field_content(0, reinterpret_cast<char*>(&l_char), _arena.get());
^
```
##########
be/test/olap/row_cursor_test.cpp:
##########
@@ -344,31 +343,31 @@
Slice l_char("well");
int32_t l_int = 10;
- left.set_field_content(0, reinterpret_cast<char*>(&l_char), _mem_pool.get());
- left.set_field_content(1, reinterpret_cast<char*>(&l_int), _mem_pool.get());
+ left.set_field_content(0, reinterpret_cast<char*>(&l_char), _arena.get());
+ left.set_field_content(1, reinterpret_cast<char*>(&l_int), _arena.get());
RowCursor right_eq;
res = right_eq.init(tablet_schema);
Slice r_char_eq("well");
int32_t r_int_eq = 10;
- right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _mem_pool.get());
- right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _mem_pool.get());
+ right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _arena.get());
+ right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _arena.get());
EXPECT_EQ(compare_row(left, right_eq), 0);
Review Comment:
warning: use of undeclared identifier 'compare_row' [clang-diagnostic-error]
```cpp
EXPECT_EQ(compare_row(left, right_eq), 0);
^
```
##########
be/test/olap/row_cursor_test.cpp:
##########
@@ -344,31 +343,31 @@
Slice l_char("well");
int32_t l_int = 10;
- left.set_field_content(0, reinterpret_cast<char*>(&l_char), _mem_pool.get());
- left.set_field_content(1, reinterpret_cast<char*>(&l_int), _mem_pool.get());
+ left.set_field_content(0, reinterpret_cast<char*>(&l_char), _arena.get());
+ left.set_field_content(1, reinterpret_cast<char*>(&l_int), _arena.get());
RowCursor right_eq;
res = right_eq.init(tablet_schema);
Slice r_char_eq("well");
int32_t r_int_eq = 10;
- right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _mem_pool.get());
- right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _mem_pool.get());
+ right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _arena.get());
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _arena.get());
^
```
##########
be/test/olap/ordered_data_compaction_test.cpp:
##########
@@ -213,15 +213,15 @@ class OrderedDataCompactionTest : public ::testing::Test {
uint32_t num_rows = 0;
for (int i = 0; i < rowset_data.size(); ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rowset_data[i].size(); ++rid) {
uint32_t c1 = std::get<0>(rowset_data[i][rid]);
uint32_t c2 = std::get<1>(rowset_data[i][rid]);
- input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &arena);
^
```
##########
be/src/olap/rowset/segment_v2/bloom_filter_index_writer.cpp:
##########
@@ -69,7 +68,7 @@ class BloomFilterIndexWriterImpl : public BloomFilterIndexWriter {
const TypeInfo* type_info)
: _bf_options(bf_options),
_type_info(type_info),
- _pool(),
+ _arena(),
Review Comment:
warning: initializer for member '_arena' is redundant [readability-redundant-member-init]
```suggestion
,
```
##########
be/test/olap/key_coder_test.cpp:
##########
@@ -22,18 +22,14 @@
#include <limits>
-#include "runtime/mem_pool.h"
#include "util/debug_util.h"
namespace doris {
class KeyCoderTest : public testing::Test {
public:
- KeyCoderTest() : _pool() {}
+ KeyCoderTest() {}
Review Comment:
warning: use '= default' to define a trivial default constructor [modernize-use-equals-default]
```suggestion
KeyCoderTest() = default;
```
##########
be/test/olap/row_cursor_test.cpp:
##########
@@ -344,31 +343,31 @@
Slice l_char("well");
int32_t l_int = 10;
- left.set_field_content(0, reinterpret_cast<char*>(&l_char), _mem_pool.get());
- left.set_field_content(1, reinterpret_cast<char*>(&l_int), _mem_pool.get());
+ left.set_field_content(0, reinterpret_cast<char*>(&l_char), _arena.get());
+ left.set_field_content(1, reinterpret_cast<char*>(&l_int), _arena.get());
RowCursor right_eq;
res = right_eq.init(tablet_schema);
Slice r_char_eq("well");
int32_t r_int_eq = 10;
- right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _mem_pool.get());
- right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _mem_pool.get());
+ right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _arena.get());
+ right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _arena.get());
EXPECT_EQ(compare_row(left, right_eq), 0);
RowCursor right_lt;
res = right_lt.init(tablet_schema);
Slice r_char_lt("well");
int32_t r_int_lt = 11;
- right_lt.set_field_content(0, reinterpret_cast<char*>(&r_char_lt), _mem_pool.get());
- right_lt.set_field_content(1, reinterpret_cast<char*>(&r_int_lt), _mem_pool.get());
+ right_lt.set_field_content(0, reinterpret_cast<char*>(&r_char_lt), _arena.get());
+ right_lt.set_field_content(1, reinterpret_cast<char*>(&r_int_lt), _arena.get());
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
right_lt.set_field_content(1, reinterpret_cast<char*>(&r_int_lt), _arena.get());
^
```
##########
be/test/olap/row_cursor_test.cpp:
##########
@@ -259,13 +258,13 @@ void set_tablet_schema_for_cmp_and_aggregate(TabletSchemaSPtr tablet_schema) {
class TestRowCursor : public testing::Test {
public:
- TestRowCursor() { _mem_pool.reset(new MemPool()); }
+ TestRowCursor() { _arena.reset(new vectorized::Arena()); }
virtual void SetUp() {}
virtual void TearDown() {}
- std::unique_ptr<MemPool> _mem_pool;
+ std::unique_ptr<Arena> _arena;
Review Comment:
warning: unknown type name 'Arena'; did you mean 'vectorized::Arena'? [clang-diagnostic-error]
```suggestion
std::unique_ptr<vectorized::Arena> _arena;
```
**be/src/vec/aggregate_functions/aggregate_function.h:34:** 'vectorized::Arena' declared here
```cpp
class Arena;
^
```
##########
be/test/olap/row_cursor_test.cpp:
##########
@@ -344,31 +343,31 @@
Slice l_char("well");
int32_t l_int = 10;
- left.set_field_content(0, reinterpret_cast<char*>(&l_char), _mem_pool.get());
- left.set_field_content(1, reinterpret_cast<char*>(&l_int), _mem_pool.get());
+ left.set_field_content(0, reinterpret_cast<char*>(&l_char), _arena.get());
+ left.set_field_content(1, reinterpret_cast<char*>(&l_int), _arena.get());
RowCursor right_eq;
res = right_eq.init(tablet_schema);
Slice r_char_eq("well");
int32_t r_int_eq = 10;
- right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _mem_pool.get());
- right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _mem_pool.get());
+ right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _arena.get());
+ right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _arena.get());
EXPECT_EQ(compare_row(left, right_eq), 0);
RowCursor right_lt;
res = right_lt.init(tablet_schema);
Slice r_char_lt("well");
int32_t r_int_lt = 11;
- right_lt.set_field_content(0, reinterpret_cast<char*>(&r_char_lt), _mem_pool.get());
- right_lt.set_field_content(1, reinterpret_cast<char*>(&r_int_lt), _mem_pool.get());
+ right_lt.set_field_content(0, reinterpret_cast<char*>(&r_char_lt), _arena.get());
+ right_lt.set_field_content(1, reinterpret_cast<char*>(&r_int_lt), _arena.get());
EXPECT_LT(compare_row(left, right_lt), 0);
Review Comment:
warning: use of undeclared identifier 'compare_row' [clang-diagnostic-error]
```cpp
EXPECT_LT(compare_row(left, right_lt), 0);
^
```
##########
be/test/olap/row_cursor_test.cpp:
##########
@@ -344,31 +343,31 @@
Slice l_char("well");
int32_t l_int = 10;
- left.set_field_content(0, reinterpret_cast<char*>(&l_char), _mem_pool.get());
- left.set_field_content(1, reinterpret_cast<char*>(&l_int), _mem_pool.get());
+ left.set_field_content(0, reinterpret_cast<char*>(&l_char), _arena.get());
+ left.set_field_content(1, reinterpret_cast<char*>(&l_int), _arena.get());
RowCursor right_eq;
res = right_eq.init(tablet_schema);
Slice r_char_eq("well");
int32_t r_int_eq = 10;
- right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _mem_pool.get());
- right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _mem_pool.get());
+ right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _arena.get());
+ right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _arena.get());
EXPECT_EQ(compare_row(left, right_eq), 0);
RowCursor right_lt;
res = right_lt.init(tablet_schema);
Slice r_char_lt("well");
int32_t r_int_lt = 11;
- right_lt.set_field_content(0, reinterpret_cast<char*>(&r_char_lt), _mem_pool.get());
- right_lt.set_field_content(1, reinterpret_cast<char*>(&r_int_lt), _mem_pool.get());
+ right_lt.set_field_content(0, reinterpret_cast<char*>(&r_char_lt), _arena.get());
+ right_lt.set_field_content(1, reinterpret_cast<char*>(&r_int_lt), _arena.get());
EXPECT_LT(compare_row(left, right_lt), 0);
RowCursor right_gt;
res = right_gt.init(tablet_schema);
Slice r_char_gt("good");
int32_t r_int_gt = 10;
- right_gt.set_field_content(0, reinterpret_cast<char*>(&r_char_gt), _mem_pool.get());
- right_gt.set_field_content(1, reinterpret_cast<char*>(&r_int_gt), _mem_pool.get());
+ right_gt.set_field_content(0, reinterpret_cast<char*>(&r_char_gt), _arena.get());
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
right_gt.set_field_content(0, reinterpret_cast<char*>(&r_char_gt), _arena.get());
^
```
##########
be/test/olap/row_cursor_test.cpp:
##########
@@ -344,31 +343,31 @@
Slice l_char("well");
int32_t l_int = 10;
- left.set_field_content(0, reinterpret_cast<char*>(&l_char), _mem_pool.get());
- left.set_field_content(1, reinterpret_cast<char*>(&l_int), _mem_pool.get());
+ left.set_field_content(0, reinterpret_cast<char*>(&l_char), _arena.get());
+ left.set_field_content(1, reinterpret_cast<char*>(&l_int), _arena.get());
RowCursor right_eq;
res = right_eq.init(tablet_schema);
Slice r_char_eq("well");
int32_t r_int_eq = 10;
- right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _mem_pool.get());
- right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _mem_pool.get());
+ right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _arena.get());
+ right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _arena.get());
EXPECT_EQ(compare_row(left, right_eq), 0);
RowCursor right_lt;
res = right_lt.init(tablet_schema);
Slice r_char_lt("well");
int32_t r_int_lt = 11;
- right_lt.set_field_content(0, reinterpret_cast<char*>(&r_char_lt), _mem_pool.get());
- right_lt.set_field_content(1, reinterpret_cast<char*>(&r_int_lt), _mem_pool.get());
+ right_lt.set_field_content(0, reinterpret_cast<char*>(&r_char_lt), _arena.get());
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
right_lt.set_field_content(0, reinterpret_cast<char*>(&r_char_lt), _arena.get());
^
```
##########
be/test/olap/row_cursor_test.cpp:
##########
@@ -344,31 +343,31 @@
Slice l_char("well");
int32_t l_int = 10;
- left.set_field_content(0, reinterpret_cast<char*>(&l_char), _mem_pool.get());
- left.set_field_content(1, reinterpret_cast<char*>(&l_int), _mem_pool.get());
+ left.set_field_content(0, reinterpret_cast<char*>(&l_char), _arena.get());
+ left.set_field_content(1, reinterpret_cast<char*>(&l_int), _arena.get());
RowCursor right_eq;
res = right_eq.init(tablet_schema);
Slice r_char_eq("well");
int32_t r_int_eq = 10;
- right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _mem_pool.get());
- right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _mem_pool.get());
+ right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _arena.get());
+ right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _arena.get());
EXPECT_EQ(compare_row(left, right_eq), 0);
RowCursor right_lt;
res = right_lt.init(tablet_schema);
Slice r_char_lt("well");
int32_t r_int_lt = 11;
- right_lt.set_field_content(0, reinterpret_cast<char*>(&r_char_lt), _mem_pool.get());
- right_lt.set_field_content(1, reinterpret_cast<char*>(&r_int_lt), _mem_pool.get());
+ right_lt.set_field_content(0, reinterpret_cast<char*>(&r_char_lt), _arena.get());
+ right_lt.set_field_content(1, reinterpret_cast<char*>(&r_int_lt), _arena.get());
EXPECT_LT(compare_row(left, right_lt), 0);
RowCursor right_gt;
res = right_gt.init(tablet_schema);
Slice r_char_gt("good");
int32_t r_int_gt = 10;
- right_gt.set_field_content(0, reinterpret_cast<char*>(&r_char_gt), _mem_pool.get());
- right_gt.set_field_content(1, reinterpret_cast<char*>(&r_int_gt), _mem_pool.get());
+ right_gt.set_field_content(0, reinterpret_cast<char*>(&r_char_gt), _arena.get());
+ right_gt.set_field_content(1, reinterpret_cast<char*>(&r_int_gt), _arena.get());
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
right_gt.set_field_content(1, reinterpret_cast<char*>(&r_int_gt), _arena.get());
^
```
##########
be/test/olap/row_cursor_test.cpp:
##########
@@ -344,31 +343,31 @@
Slice l_char("well");
int32_t l_int = 10;
- left.set_field_content(0, reinterpret_cast<char*>(&l_char), _mem_pool.get());
- left.set_field_content(1, reinterpret_cast<char*>(&l_int), _mem_pool.get());
+ left.set_field_content(0, reinterpret_cast<char*>(&l_char), _arena.get());
+ left.set_field_content(1, reinterpret_cast<char*>(&l_int), _arena.get());
RowCursor right_eq;
res = right_eq.init(tablet_schema);
Slice r_char_eq("well");
int32_t r_int_eq = 10;
- right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _mem_pool.get());
- right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _mem_pool.get());
+ right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _arena.get());
+ right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _arena.get());
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _arena.get());
^
```
##########
be/test/olap/rowset/segment_v2/binary_dict_page_test.cpp:
##########
@@ -95,7 +94,7 @@ class BinaryDictPageTest : public testing::Test {
EXPECT_EQ(slices.size(), page_decoder.count());
//check values
- MemPool pool;
+ vectorized::Arena pool;
auto type_info = get_scalar_type_info(OLAP_FIELD_TYPE_VARCHAR);
size_t size = slices.size();
std::unique_ptr<ColumnVectorBatch> cvb;
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
std::unique_ptr<ColumnVectorBatch> cvb;
^
```
##########
be/test/olap/rowset/segment_v2/binary_dict_page_test.cpp:
##########
@@ -199,7 +198,7 @@
EXPECT_TRUE(status.ok());
//check values
- MemPool pool;
+ vectorized::Arena pool;
auto type_info = get_scalar_type_info(OLAP_FIELD_TYPE_VARCHAR);
std::unique_ptr<ColumnVectorBatch> cvb;
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
std::unique_ptr<ColumnVectorBatch> cvb;
^
```
##########
be/test/olap/rowset/segment_v2/binary_plain_page_test.cpp:
##########
@@ -70,7 +69,7 @@ class BinaryPlainPageTest : public testing::Test {
EXPECT_TRUE(status.ok());
//test1
- MemPool pool;
+ vectorized::Arena pool;
size_t size = 3;
std::unique_ptr<ColumnVectorBatch> cvb;
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
std::unique_ptr<ColumnVectorBatch> cvb;
^
```
##########
be/test/olap/rowset/segment_v2/binary_dict_page_test.cpp:
##########
@@ -199,7 +198,7 @@
EXPECT_TRUE(status.ok());
//check values
- MemPool pool;
+ vectorized::Arena pool;
auto type_info = get_scalar_type_info(OLAP_FIELD_TYPE_VARCHAR);
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(1, false, type_info, nullptr, &cvb);
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
ColumnVectorBatch::create(1, false, type_info, nullptr, &cvb);
^
```
##########
be/test/olap/row_cursor_test.cpp:
##########
@@ -344,31 +343,31 @@
Slice l_char("well");
int32_t l_int = 10;
- left.set_field_content(0, reinterpret_cast<char*>(&l_char), _mem_pool.get());
- left.set_field_content(1, reinterpret_cast<char*>(&l_int), _mem_pool.get());
+ left.set_field_content(0, reinterpret_cast<char*>(&l_char), _arena.get());
+ left.set_field_content(1, reinterpret_cast<char*>(&l_int), _arena.get());
RowCursor right_eq;
res = right_eq.init(tablet_schema);
Slice r_char_eq("well");
int32_t r_int_eq = 10;
- right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _mem_pool.get());
- right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _mem_pool.get());
+ right_eq.set_field_content(0, reinterpret_cast<char*>(&r_char_eq), _arena.get());
+ right_eq.set_field_content(1, reinterpret_cast<char*>(&r_int_eq), _arena.get());
EXPECT_EQ(compare_row(left, right_eq), 0);
RowCursor right_lt;
res = right_lt.init(tablet_schema);
Slice r_char_lt("well");
int32_t r_int_lt = 11;
- right_lt.set_field_content(0, reinterpret_cast<char*>(&r_char_lt), _mem_pool.get());
- right_lt.set_field_content(1, reinterpret_cast<char*>(&r_int_lt), _mem_pool.get());
+ right_lt.set_field_content(0, reinterpret_cast<char*>(&r_char_lt), _arena.get());
+ right_lt.set_field_content(1, reinterpret_cast<char*>(&r_int_lt), _arena.get());
EXPECT_LT(compare_row(left, right_lt), 0);
RowCursor right_gt;
res = right_gt.init(tablet_schema);
Slice r_char_gt("good");
int32_t r_int_gt = 10;
- right_gt.set_field_content(0, reinterpret_cast<char*>(&r_char_gt), _mem_pool.get());
- right_gt.set_field_content(1, reinterpret_cast<char*>(&r_int_gt), _mem_pool.get());
+ right_gt.set_field_content(0, reinterpret_cast<char*>(&r_char_gt), _arena.get());
+ right_gt.set_field_content(1, reinterpret_cast<char*>(&r_int_gt), _arena.get());
EXPECT_GT(compare_row(left, right_gt), 0);
Review Comment:
warning: use of undeclared identifier 'compare_row' [clang-diagnostic-error]
```cpp
EXPECT_GT(compare_row(left, right_gt), 0);
^
```
##########
be/test/olap/rowset/segment_v2/binary_plain_page_test.cpp:
##########
@@ -70,7 +69,7 @@
EXPECT_TRUE(status.ok());
//test1
- MemPool pool;
+ vectorized::Arena pool;
size_t size = 3;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(size, true, get_scalar_type_info(OLAP_FIELD_TYPE_VARCHAR),
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
ColumnVectorBatch::create(size, true, get_scalar_type_info(OLAP_FIELD_TYPE_VARCHAR),
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1482623017
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1484892711
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1487833732
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1151477287
##########
be/src/vec/data_types/data_type_bitmap.h:
##########
@@ -72,7 +72,7 @@ class DataTypeBitMap : public IDataType {
}
void to_string(const IColumn& column, size_t row_num, BufferWritable& ostr) const override;
- [[noreturn]] virtual Field get_default() const override {
+ virtual Field get_default() const override {
Review Comment:
warning: 'virtual' is redundant since the function is already declared 'override' [modernize-use-override]
```suggestion
Field get_default() const override {
```
##########
be/src/vec/data_types/data_type_quantilestate.h:
##########
@@ -70,7 +70,7 @@ class DataTypeQuantileState : public IDataType {
}
void to_string(const IColumn& column, size_t row_num, BufferWritable& ostr) const override;
- [[noreturn]] virtual Field get_default() const override {
+ virtual Field get_default() const override {
Review Comment:
warning: 'virtual' is redundant since the function is already declared 'override' [modernize-use-override]
```suggestion
Field get_default() const override {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1487971733
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1485397341
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1138485090
##########
be/src/olap/memtable.h:
##########
@@ -48,7 +49,8 @@ class MemTable {
int64_t tablet_id() const { return _tablet->tablet_id(); }
size_t memory_usage() const {
- return _insert_mem_tracker->consumption() + _flush_mem_tracker->consumption();
+ return _insert_mem_tracker->consumption() + _arena->size() +
Review Comment:
same as above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1484106630
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1484425943
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1138391455
##########
be/src/olap/rowset/segment_v2/bloom_filter_index_writer.cpp:
##########
@@ -144,7 +142,7 @@ class BloomFilterIndexWriterImpl : public BloomFilterIndexWriter {
uint64_t size() override {
uint64_t total_size = _bf_buffer_size;
- total_size += _pool.total_allocated_bytes();
Review Comment:
same problem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1484036343
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1488097763
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1138390962
##########
be/src/olap/rowset/segment_v2/bitmap_index_writer.cpp:
##########
@@ -169,7 +169,7 @@ class BitmapIndexWriterImpl : public BitmapIndexWriter {
size += _null_bitmap.getSizeInBytes(false);
size += _reverted_index_size;
size += _mem_index.size() * sizeof(CppType);
- size += _pool.total_allocated_bytes();
Review Comment:
same problem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1470026888
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1471773999
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1147452797
##########
be/src/olap/memtable.h:
##########
@@ -48,7 +49,8 @@ class MemTable {
int64_t tablet_id() const { return _tablet->tablet_id(); }
size_t memory_usage() const {
- return _insert_mem_tracker->consumption() + _flush_mem_tracker->consumption();
+ return _insert_mem_tracker->consumption() + _arena->allocated_size() +
Review Comment:
warning: no member named 'allocated_size' in 'doris::vectorized::Arena' [clang-diagnostic-error]
```cpp
return _insert_mem_tracker->consumption() + _arena->allocated_size() +
^
```
##########
be/src/olap/rowset/segment_v2/binary_dict_page.cpp:
##########
@@ -152,7 +151,7 @@ size_t BinaryDictPageBuilder::count() const {
}
uint64_t BinaryDictPageBuilder::size() const {
- return _pool.total_allocated_bytes() + _data_page_builder->size();
+ return _arena.allocated_size() + _data_page_builder->size();
Review Comment:
warning: no member named 'allocated_size' in 'doris::vectorized::Arena' [clang-diagnostic-error]
```cpp
return _arena.allocated_size() + _data_page_builder->size();
^
```
##########
be/src/olap/rowset/segment_v2/bitmap_index_writer.cpp:
##########
@@ -169,7 +169,7 @@ class BitmapIndexWriterImpl : public BitmapIndexWriter {
size += _null_bitmap.getSizeInBytes(false);
size += _reverted_index_size;
size += _mem_index.size() * sizeof(CppType);
- size += _pool.total_allocated_bytes();
+ size += _arena.allocated_size();
Review Comment:
warning: no member named 'allocated_size' in 'doris::vectorized::Arena' [clang-diagnostic-error]
```cpp
size += _arena.allocated_size();
^
```
##########
be/src/olap/rowset/segment_v2/bloom_filter_index_writer.cpp:
##########
@@ -144,7 +142,7 @@ class BloomFilterIndexWriterImpl : public BloomFilterIndexWriter {
uint64_t size() override {
uint64_t total_size = _bf_buffer_size;
- total_size += _pool.total_allocated_bytes();
+ total_size += _arena.allocated_size();
Review Comment:
warning: no member named 'allocated_size' in 'doris::vectorized::Arena' [clang-diagnostic-error]
```cpp
total_size += _arena.allocated_size();
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #17820:
URL: https://github.com/apache/doris/pull/17820
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1484091739
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1472968796
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1138488922
##########
be/src/olap/rowset/segment_v2/bloom_filter_index_writer.cpp:
##########
@@ -144,7 +142,7 @@ class BloomFilterIndexWriterImpl : public BloomFilterIndexWriter {
uint64_t size() override {
uint64_t total_size = _bf_buffer_size;
- total_size += _pool.total_allocated_bytes();
Review Comment:
same as above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1138481622
##########
be/src/olap/memtable.cpp:
##########
@@ -64,10 +64,10 @@ MemTable::MemTable(TabletSharedPtr tablet, Schema* schema, const TabletSchema* t
_insert_mem_tracker_use_hook = std::make_unique<MemTracker>(
fmt::format("MemTableHookInsert:TabletId={}", std::to_string(tablet_id())));
#endif
- _table_mem_pool = std::make_unique<MemPool>(_insert_mem_tracker.get());
Review Comment:
Arena add `allocated_size() `, `MemTable::memory_usage()` contains `arena->allocated_size()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1478953003
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1147334110
##########
be/src/vec/exec/vaggregation_node.h:
##########
@@ -96,8 +96,10 @@ struct AggregationMethodSerialized {
_arena.reset();
if (total_bytes > _serialized_key_buffer_size) {
_serialized_key_buffer_size = total_bytes;
- _mem_pool->clear();
Review Comment:
use arena on line 89 and not add _serialize_key_arena
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1488008360
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1138035579
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -395,14 +394,14 @@ TEST_F(SegCompactionTest, SegCompactionInterleaveWithBig_ooooOOoOooooooooO) {
num_segments = 1;
rows_per_segment = 4096;
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
+ input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &arena);
s = rowset_writer->add_row(input_row);
Review Comment:
warning: no member named 'add_row' in 'doris::RowsetWriter' [clang-diagnostic-error]
```cpp
s = rowset_writer->add_row(input_row);
^
```
##########
be/test/vec/olap/vertical_compaction_test.cpp:
##########
@@ -205,15 +205,15 @@
uint32_t num_rows = 0;
for (int i = 0; i < rowset_data.size(); ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rowset_data[i].size(); ++rid) {
uint32_t c1 = std::get<0>(rowset_data[i][rid]);
uint32_t c2 = std::get<1>(rowset_data[i][rid]);
- input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &arena);
^
```
##########
be/test/vec/olap/vertical_compaction_test.cpp:
##########
@@ -205,15 +205,15 @@
uint32_t num_rows = 0;
for (int i = 0; i < rowset_data.size(); ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rowset_data[i].size(); ++rid) {
uint32_t c1 = std::get<0>(rowset_data[i][rid]);
uint32_t c2 = std::get<1>(rowset_data[i][rid]);
- input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &arena);
if (tablet_schema->keys_type() == UNIQUE_KEYS) {
uint8_t num = 0;
- input_row.set_field_content(2, reinterpret_cast<char*>(&num), &mem_pool);
+ input_row.set_field_content(2, reinterpret_cast<char*>(&num), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(2, reinterpret_cast<char*>(&num), &arena);
^
```
##########
be/test/vec/olap/vertical_compaction_test.cpp:
##########
@@ -205,15 +205,15 @@
uint32_t num_rows = 0;
for (int i = 0; i < rowset_data.size(); ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rowset_data[i].size(); ++rid) {
uint32_t c1 = std::get<0>(rowset_data[i][rid]);
uint32_t c2 = std::get<1>(rowset_data[i][rid]);
- input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &arena);
if (tablet_schema->keys_type() == UNIQUE_KEYS) {
uint8_t num = 0;
- input_row.set_field_content(2, reinterpret_cast<char*>(&num), &mem_pool);
+ input_row.set_field_content(2, reinterpret_cast<char*>(&num), &arena);
}
s = rowset_writer->add_row(input_row);
Review Comment:
warning: no member named 'add_row' in 'doris::RowsetWriter' [clang-diagnostic-error]
```cpp
s = rowset_writer->add_row(input_row);
^
```
##########
be/test/vec/olap/vertical_compaction_test.cpp:
##########
@@ -205,15 +205,15 @@ class VerticalCompactionTest : public ::testing::Test {
uint32_t num_rows = 0;
for (int i = 0; i < rowset_data.size(); ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rowset_data[i].size(); ++rid) {
uint32_t c1 = std::get<0>(rowset_data[i][rid]);
uint32_t c2 = std::get<1>(rowset_data[i][rid]);
- input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&c2), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(0, reinterpret_cast<char*>(&c1), &arena);
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1136898526
##########
be/test/olap/key_coder_test.cpp:
##########
@@ -22,18 +22,14 @@
#include <limits>
-#include "runtime/mem_pool.h"
#include "util/debug_util.h"
namespace doris {
class KeyCoderTest : public testing::Test {
public:
- KeyCoderTest() : _pool() {}
- virtual ~KeyCoderTest() {}
-
-private:
- MemPool _pool;
+ KeyCoderTest() = default;
+ virtual ~KeyCoderTest() = default;
Review Comment:
warning: prefer using 'override' or (rarely) 'final' instead of 'virtual' [modernize-use-override]
```suggestion
~KeyCoderTest() override = default;
```
##########
be/test/olap/rowset/segment_v2/rle_page_test.cpp:
##########
@@ -37,7 +36,7 @@ class RlePageTest : public testing::Test {
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
std::unique_ptr<ColumnVectorBatch> cvb;
^
```
##########
be/test/olap/rowset/segment_v2/rle_page_test.cpp:
##########
@@ -37,7 +36,7 @@
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
^
```
##########
be/test/olap/rowset/segment_v2/rle_page_test.cpp:
##########
@@ -37,7 +36,7 @@
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
ColumnBlock block(cvb.get(), &pool);
Review Comment:
warning: unknown type name 'ColumnBlock' [clang-diagnostic-error]
```cpp
ColumnBlock block(cvb.get(), &pool);
^
```
##########
be/test/olap/rowset/segment_v2/rle_page_test.cpp:
##########
@@ -74,7 +73,7 @@
EXPECT_EQ(0, rle_page_decoder.current_index());
EXPECT_EQ(size, rle_page_decoder.count());
- MemPool pool;
+ vectorized::Arena pool;
Review Comment:
warning: variable has incomplete type 'vectorized::Arena' [clang-diagnostic-error]
```cpp
vectorized::Arena pool;
^
```
**be/src/vec/columns/column.h:60:** forward declaration of 'doris::vectorized::Arena'
```cpp
class Arena;
^
```
##########
be/test/olap/rowset/segment_v2/rle_page_test.cpp:
##########
@@ -37,7 +36,7 @@
template <FieldType type, class PageDecoderType>
void copy_one(PageDecoderType* decoder, typename TypeTraits<type>::CppType* ret) {
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
Review Comment:
warning: use of undeclared identifier 'get_scalar_type_info' [clang-diagnostic-error]
```cpp
ColumnVectorBatch::create(1, true, get_scalar_type_info(type), nullptr, &cvb);
^
```
##########
be/test/olap/rowset/segment_v2/rle_page_test.cpp:
##########
@@ -74,7 +73,7 @@
EXPECT_EQ(0, rle_page_decoder.current_index());
EXPECT_EQ(size, rle_page_decoder.count());
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
std::unique_ptr<ColumnVectorBatch> cvb;
^
```
##########
be/test/olap/rowset/segment_v2/rle_page_test.cpp:
##########
@@ -74,7 +73,7 @@
EXPECT_EQ(0, rle_page_decoder.current_index());
EXPECT_EQ(size, rle_page_decoder.count());
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(size, true, get_scalar_type_info(Type), nullptr, &cvb);
Review Comment:
warning: use of undeclared identifier 'ColumnVectorBatch' [clang-diagnostic-error]
```cpp
ColumnVectorBatch::create(size, true, get_scalar_type_info(Type), nullptr, &cvb);
^
```
##########
be/test/olap/rowset/segment_v2/rle_page_test.cpp:
##########
@@ -74,7 +73,7 @@
EXPECT_EQ(0, rle_page_decoder.current_index());
EXPECT_EQ(size, rle_page_decoder.count());
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(size, true, get_scalar_type_info(Type), nullptr, &cvb);
Review Comment:
warning: use of undeclared identifier 'get_scalar_type_info' [clang-diagnostic-error]
```cpp
ColumnVectorBatch::create(size, true, get_scalar_type_info(Type), nullptr, &cvb);
^
```
##########
be/test/olap/rowset/segment_v2/rle_page_test.cpp:
##########
@@ -74,7 +73,7 @@
EXPECT_EQ(0, rle_page_decoder.current_index());
EXPECT_EQ(size, rle_page_decoder.count());
- MemPool pool;
+ vectorized::Arena pool;
std::unique_ptr<ColumnVectorBatch> cvb;
ColumnVectorBatch::create(size, true, get_scalar_type_info(Type), nullptr, &cvb);
ColumnBlock block(cvb.get(), &pool);
Review Comment:
warning: unknown type name 'ColumnBlock' [clang-diagnostic-error]
```cpp
ColumnBlock block(cvb.get(), &pool);
^
```
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -251,14 +250,14 @@ TEST_F(SegCompactionTest, SegCompactionThenRead) {
// k2 := k1 * 10
// k3 := rid
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
^
```
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -361,14 +360,14 @@
int num_segments = 4;
uint32_t rows_per_segment = 4096;
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
+ input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &arena);
^
```
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -361,14 +360,14 @@
int num_segments = 4;
uint32_t rows_per_segment = 4096;
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
^
```
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -251,14 +250,14 @@
// k2 := k1 * 10
// k3 := rid
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
^
```
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -251,14 +250,14 @@
// k2 := k1 * 10
// k3 := rid
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
+ input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &arena);
s = rowset_writer->add_row(input_row);
Review Comment:
warning: no member named 'add_row' in 'doris::RowsetWriter' [clang-diagnostic-error]
```cpp
s = rowset_writer->add_row(input_row);
^
```
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -251,14 +250,14 @@
// k2 := k1 * 10
// k3 := rid
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
+ input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &arena);
^
```
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -361,14 +360,14 @@
int num_segments = 4;
uint32_t rows_per_segment = 4096;
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
+ input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &arena);
s = rowset_writer->add_row(input_row);
Review Comment:
warning: no member named 'add_row' in 'doris::RowsetWriter' [clang-diagnostic-error]
```cpp
s = rowset_writer->add_row(input_row);
^
```
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -378,14 +377,14 @@
num_segments = 2;
rows_per_segment = 6400;
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
^
```
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -361,14 +360,14 @@
int num_segments = 4;
uint32_t rows_per_segment = 4096;
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
^
```
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -378,14 +377,14 @@
num_segments = 2;
rows_per_segment = 6400;
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
^
```
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -378,14 +377,14 @@
num_segments = 2;
rows_per_segment = 6400;
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
+ input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &arena);
s = rowset_writer->add_row(input_row);
Review Comment:
warning: no member named 'add_row' in 'doris::RowsetWriter' [clang-diagnostic-error]
```cpp
s = rowset_writer->add_row(input_row);
^
```
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -378,14 +377,14 @@
num_segments = 2;
rows_per_segment = 6400;
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
+ input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &arena);
^
```
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -395,14 +394,14 @@
num_segments = 1;
rows_per_segment = 4096;
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
^
```
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -395,14 +394,14 @@
num_segments = 1;
rows_per_segment = 4096;
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
^
```
##########
be/test/olap/segcompaction_test.cpp:
##########
@@ -395,14 +394,14 @@
num_segments = 1;
rows_per_segment = 4096;
for (int i = 0; i < num_segments; ++i) {
- MemPool mem_pool;
+ vectorized::Arena arena;
for (int rid = 0; rid < rows_per_segment; ++rid) {
uint32_t k1 = rid * 100 + i;
uint32_t k2 = i;
uint32_t k3 = rid;
- input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &mem_pool);
- input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &mem_pool);
- input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &mem_pool);
+ input_row.set_field_content(0, reinterpret_cast<char*>(&k1), &arena);
+ input_row.set_field_content(1, reinterpret_cast<char*>(&k2), &arena);
+ input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &arena);
Review Comment:
warning: no member named 'set_field_content' in 'doris::RowCursor' [clang-diagnostic-error]
```cpp
input_row.set_field_content(2, reinterpret_cast<char*>(&k3), &arena);
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1144278271
##########
be/src/vec/common/arena.h:
##########
@@ -130,8 +133,10 @@ class Arena : private boost::noncopyable {
size_t linear_growth_threshold_ = 128 * 1024 * 1024)
: growth_factor(growth_factor_),
linear_growth_threshold(linear_growth_threshold_),
+ initial_size(initial_size_),
Review Comment:
warning: field 'initial_size' will be initialized after field 'growth_factor' [clang-diagnostic-reorder-ctor]
```suggestion
: growth_factor(growth_factor_),
linear_growth_threshold(linear_growth_threshold_),
initial_size(initial_size_),
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1479091201
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1147308885
##########
be/src/vec/common/arena.h:
##########
@@ -271,9 +276,17 @@ class Arena : private boost::noncopyable {
return res;
}
+ void clear() {
+ delete head;
+ head = new Chunk(initial_size, nullptr), size_in_bytes = head->size(),
+ size_in_allocated = 0;
+ }
+
/// Size of chunks in bytes.
size_t size() const { return size_in_bytes; }
+ size_t allocated_size() const { return size_in_allocated + head->remaining(); }
Review Comment:
pos-start
##########
be/src/vec/common/arena.h:
##########
@@ -84,10 +84,12 @@ class Arena : private boost::noncopyable {
size_t growth_factor;
size_t linear_growth_threshold;
+ size_t initial_size;
/// Last contiguous chunk of memory.
Chunk* head;
size_t size_in_bytes;
+ size_t size_in_allocated;
Review Comment:
add comment for this variable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1478977073
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1471198914
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1477347469
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1479928062
TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 36 seconds
stream load tsv: 475 seconds loaded 74807831229 Bytes, about 150 MB/s
stream load json: 24 seconds loaded 2358488459 Bytes, about 93 MB/s
stream load orc: 75 seconds loaded 1101869774 Bytes, about 14 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230322165333_clickbench_pr_118790.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1487816013
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1151310671
##########
be/src/olap/predicate_creator.h:
##########
@@ -51,7 +51,7 @@ class IntegerPredicateCreator : public PredicateCreator<ConditionType> {
public:
using CppType = typename PredicatePrimitiveTypeTraits<Type>::PredicateFieldType;
ColumnPredicate* create(const TabletColumn& column, int index, const ConditionType& conditions,
- bool opposite, MemPool* pool) override {
+ bool opposite, vectorized::Arena* arena) override {
if constexpr (PredicateTypeTraits::is_list(PT)) {
return create_in_list_predicate<Type, PT, ConditionType, decltype(convert)>(
Review Comment:
warning: no matching function for call to 'create_in_list_predicate' [clang-diagnostic-error]
```cpp
return create_in_list_predicate<Type, PT, ConditionType, decltype(convert)>(
^
```
**/usr/include/c++/11/bits/unique_ptr.h:961:** in instantiation of member function 'doris::IntegerPredicateCreator<doris::TYPE_TINYINT, doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>::create' requested here
```cpp
{ return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
^
```
**be/src/olap/predicate_creator.h:160:** in instantiation of function template specialization 'std::make_unique<doris::IntegerPredicateCreator<doris::TYPE_TINYINT, doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>>' requested here
```cpp
return std::make_unique<IntegerPredicateCreator<TYPE_TINYINT, PT, ConditionType>>();
^
```
**be/src/olap/predicate_creator.h:245:** in instantiation of function template specialization 'doris::get_creator<doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>' requested here
```cpp
return get_creator<PT, ConditionType>(column.type())
^
```
**be/src/olap/predicate_creator.h:262:** in instantiation of function template specialization 'doris::create_predicate<doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>' requested here
```cpp
return create_predicate<PT, std::vector<std::string>>(column, index, conditions, opposite,
^
```
**be/src/olap/predicate_creator.h:290:** in instantiation of function template specialization 'doris::create_list_predicate<doris::PredicateType::IN_LIST>' requested here
```cpp
create = create_list_predicate<PredicateType::IN_LIST>;
^
```
**be/src/olap/in_list_predicate.h:676:** candidate function template not viable: requires at most 3 arguments, but 4 were provided
```cpp
ColumnPredicate* create_in_list_predicate(uint32_t column_id,
^
```
##########
be/src/olap/predicate_creator.h:
##########
@@ -81,7 +81,7 @@
public:
using CppType = typename PredicatePrimitiveTypeTraits<Type>::PredicateFieldType;
ColumnPredicate* create(const TabletColumn& column, int index, const ConditionType& conditions,
- bool opposite, MemPool* pool) override {
+ bool opposite, vectorized::Arena* arena) override {
if constexpr (PredicateTypeTraits::is_list(PT)) {
return create_in_list_predicate<Type, PT, ConditionType, decltype(convert)>(
Review Comment:
warning: no matching function for call to 'create_in_list_predicate' [clang-diagnostic-error]
```cpp
return create_in_list_predicate<Type, PT, ConditionType, decltype(convert)>(
^
```
**/usr/include/c++/11/bits/unique_ptr.h:961:** in instantiation of member function 'doris::DecimalPredicateCreator<doris::TYPE_DECIMAL32, doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>::create' requested here
```cpp
{ return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
^
```
**be/src/olap/predicate_creator.h:189:** in instantiation of function template specialization 'std::make_unique<doris::DecimalPredicateCreator<doris::TYPE_DECIMAL32, doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>>' requested here
```cpp
return std::make_unique<DecimalPredicateCreator<TYPE_DECIMAL32, PT, ConditionType>>();
^
```
**be/src/olap/predicate_creator.h:245:** in instantiation of function template specialization 'doris::get_creator<doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>' requested here
```cpp
return get_creator<PT, ConditionType>(column.type())
^
```
**be/src/olap/predicate_creator.h:262:** in instantiation of function template specialization 'doris::create_predicate<doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>' requested here
```cpp
return create_predicate<PT, std::vector<std::string>>(column, index, conditions, opposite,
^
```
**be/src/olap/predicate_creator.h:290:** in instantiation of function template specialization 'doris::create_list_predicate<doris::PredicateType::IN_LIST>' requested here
```cpp
create = create_list_predicate<PredicateType::IN_LIST>;
^
```
**be/src/olap/in_list_predicate.h:676:** candidate function template not viable: requires at most 3 arguments, but 5 were provided
```cpp
ColumnPredicate* create_in_list_predicate(uint32_t column_id,
^
```
##########
be/src/olap/predicate_creator.h:
##########
@@ -105,26 +105,26 @@
class StringPredicateCreator : public PredicateCreator<ConditionType> {
public:
ColumnPredicate* create(const TabletColumn& column, int index, const ConditionType& conditions,
- bool opposite, MemPool* pool) override {
+ bool opposite, vectorized::Arena* arena) override {
if constexpr (PredicateTypeTraits::is_list(PT)) {
return create_in_list_predicate<Type, PT, ConditionType, decltype(convert)>(
Review Comment:
warning: no matching function for call to 'create_in_list_predicate' [clang-diagnostic-error]
```cpp
return create_in_list_predicate<Type, PT, ConditionType, decltype(convert)>(
^
```
**/usr/include/c++/11/bits/unique_ptr.h:961:** in instantiation of member function 'doris::StringPredicateCreator<doris::TYPE_CHAR, doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>::create' requested here
```cpp
{ return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
^
```
**be/src/olap/predicate_creator.h:198:** in instantiation of function template specialization 'std::make_unique<doris::StringPredicateCreator<doris::TYPE_CHAR, doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>>' requested here
```cpp
return std::make_unique<StringPredicateCreator<TYPE_CHAR, PT, ConditionType>>();
^
```
**be/src/olap/predicate_creator.h:245:** in instantiation of function template specialization 'doris::get_creator<doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>' requested here
```cpp
return get_creator<PT, ConditionType>(column.type())
^
```
**be/src/olap/predicate_creator.h:262:** in instantiation of function template specialization 'doris::create_predicate<doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>' requested here
```cpp
return create_predicate<PT, std::vector<std::string>>(column, index, conditions, opposite,
^
```
**be/src/olap/predicate_creator.h:290:** in instantiation of function template specialization 'doris::create_list_predicate<doris::PredicateType::IN_LIST>' requested here
```cpp
create = create_list_predicate<PredicateType::IN_LIST>;
^
```
**be/src/olap/in_list_predicate.h:676:** candidate function template not viable: requires at most 3 arguments, but 6 were provided
```cpp
ColumnPredicate* create_in_list_predicate(uint32_t column_id,
^
```
##########
be/src/olap/predicate_creator.h:
##########
@@ -140,7 +140,7 @@
: _convert(convert) {}
ColumnPredicate* create(const TabletColumn& column, int index, const ConditionType& conditions,
- bool opposite, MemPool* pool) override {
+ bool opposite, vectorized::Arena* arena) override {
if constexpr (PredicateTypeTraits::is_list(PT)) {
return create_in_list_predicate<Type, PT, ConditionType, decltype(_convert)>(
Review Comment:
warning: no matching function for call to 'create_in_list_predicate' [clang-diagnostic-error]
```cpp
return create_in_list_predicate<Type, PT, ConditionType, decltype(_convert)>(
^
```
**be/src/olap/predicate_creator.h:138:** in instantiation of member function 'doris::CustomPredicateCreator<doris::TYPE_DECIMALV2, doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>::create' requested here
```cpp
CustomPredicateCreator(const std::function<CppType(const std::string& condition)>& convert)
^
```
**/usr/include/c++/11/bits/unique_ptr.h:961:** in instantiation of member function 'doris::CustomPredicateCreator<doris::TYPE_DECIMALV2, doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>::CustomPredicateCreator' requested here
```cpp
{ return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
^
```
**be/src/olap/predicate_creator.h:181:** in instantiation of function template specialization 'std::make_unique<doris::CustomPredicateCreator<doris::TYPE_DECIMALV2, doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>, (lambda at /github/workspace/be/src/olap/predicate_creator.h:183:17)>' requested here
```cpp
return std::make_unique<CustomPredicateCreator<TYPE_DECIMALV2, PT, ConditionType>>(
^
```
**be/src/olap/predicate_creator.h:245:** in instantiation of function template specialization 'doris::get_creator<doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>' requested here
```cpp
return get_creator<PT, ConditionType>(column.type())
^
```
**be/src/olap/predicate_creator.h:262:** in instantiation of function template specialization 'doris::create_predicate<doris::PredicateType::IN_LIST, std::vector<std::basic_string<char>>>' requested here
```cpp
return create_predicate<PT, std::vector<std::string>>(column, index, conditions, opposite,
^
```
**be/src/olap/predicate_creator.h:290:** in instantiation of function template specialization 'doris::create_list_predicate<doris::PredicateType::IN_LIST>' requested here
```cpp
create = create_list_predicate<PredicateType::IN_LIST>;
^
```
**be/src/olap/in_list_predicate.h:676:** candidate function template not viable: requires at most 3 arguments, but 4 were provided
```cpp
ColumnPredicate* create_in_list_predicate(uint32_t column_id,
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1138368362
##########
be/src/olap/memtable.cpp:
##########
@@ -64,10 +64,10 @@ MemTable::MemTable(TabletSharedPtr tablet, Schema* schema, const TabletSchema* t
_insert_mem_tracker_use_hook = std::make_unique<MemTracker>(
fmt::format("MemTableHookInsert:TabletId={}", std::to_string(tablet_id())));
#endif
- _table_mem_pool = std::make_unique<MemPool>(_insert_mem_tracker.get());
Review Comment:
insert mem tracker is lost???
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1138372090
##########
be/src/olap/memtable.h:
##########
@@ -48,7 +49,8 @@ class MemTable {
int64_t tablet_id() const { return _tablet->tablet_id(); }
size_t memory_usage() const {
- return _insert_mem_tracker->consumption() + _flush_mem_tracker->consumption();
+ return _insert_mem_tracker->consumption() + _arena->size() +
Review Comment:
If arena allocated some unused memory, then the memory usage maybe larger than actual usage. It will flush out many small segments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1138387314
##########
be/src/olap/rowset/segment_v2/binary_dict_page.cpp:
##########
@@ -152,7 +151,7 @@ size_t BinaryDictPageBuilder::count() const {
}
uint64_t BinaryDictPageBuilder::size() const {
- return _pool.total_allocated_bytes() + _data_page_builder->size();
+ return _arena.size() + _data_page_builder->size();
Review Comment:
the same problem. The arena's allocated size maybe very large than the data's actual size. The segment maybe very small.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1138488477
##########
be/src/olap/rowset/segment_v2/bitmap_index_writer.cpp:
##########
@@ -169,7 +169,7 @@ class BitmapIndexWriterImpl : public BitmapIndexWriter {
size += _null_bitmap.getSizeInBytes(false);
size += _reverted_index_size;
size += _mem_index.size() * sizeof(CppType);
- size += _pool.total_allocated_bytes();
Review Comment:
same as above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1147323045
##########
be/src/vec/columns/predicate_column.h:
##########
@@ -340,8 +340,8 @@ class PredicateColumnType final : public COWHelper<IColumn, PredicateColumnType<
void clear() override {
data.clear();
- if (_pool != nullptr) {
- _pool->clear();
+ if (_arena != nullptr) {
Review Comment:
juse reset arena pointer and create a new one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #17820:
URL: https://github.com/apache/doris/pull/17820#discussion_r1147334110
##########
be/src/vec/exec/vaggregation_node.h:
##########
@@ -96,8 +96,10 @@ struct AggregationMethodSerialized {
_arena.reset();
if (total_bytes > _serialized_key_buffer_size) {
_serialized_key_buffer_size = total_bytes;
- _mem_pool->clear();
Review Comment:
create a new arena
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #17820: [enhancement](memory) Remove MemPool and replace it with Arena
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #17820:
URL: https://github.com/apache/doris/pull/17820#issuecomment-1488020499
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org