[GitHub] [doris] dataroaring commented on a diff in pull request #18686: [performance](stream-load) use vector instead of skiplist when insert dup keys

["dataroaring (via GitHub)" <gi...@apache.org>]

dataroaring commented on code in PR #18686:
URL: https://github.com/apache/doris/pull/18686#discussion_r1172094084


##########
be/src/olap/memtable.cpp:
##########
@@ -243,11 +242,22 @@ void MemTable::_collect_vskiplist_results() {
     VecTable::Iterator it(_vec_skip_list.get());
     vectorized::Block in_block = _input_mutable_block.to_block();
     if (_keys_type == KeysType::DUP_KEYS) {
+        vectorized::MutableBlock mutable_block =
+                vectorized::MutableBlock::build_mutable_block(&in_block);
+        _vec_row_comparator->set_block(&mutable_block);
+        std::sort(_row_in_blocks.begin(), _row_in_blocks.end(),
+                  [this](const RowInBlock* l, const RowInBlock* r) -> bool {
+                      if ((*(this->_vec_row_comparator))(l, r) == 0) {
+                          return l->_row_pos - r->_row_pos > 0;
+                      } else {
+                          return (*(this->_vec_row_comparator))(l, r) < 0;

Review Comment:
   two times comparator?



##########
be/src/olap/memtable.cpp:
##########
@@ -191,8 +191,7 @@ void MemTable::_insert_one_row_from_block(RowInBlock* row_in_block) {
     _rows++;
     bool overwritten = false;
     if (_keys_type == KeysType::DUP_KEYS) {
-        // TODO: dup keys only need sort opertaion. Rethink skiplist is the beat way to sort columns?
-        _vec_skip_list->Insert(row_in_block, &overwritten);
+        // dup keys store in vector _row_in_blocks and sort it on flush stage

Review Comment:
   store row_in_blocks in vector.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org